2 Comments »Posted on Sunday 10 May 2009 at 5:51 pm by Jacob Aron
In Getting It Wrong, Health & Medicine

The Daily Mail have reported that the Government are indoctrinating children into supporting the MMR vaccine. It seems that in the January 2008 Biology GCSE paper pupils were awarded marks for criticising the controversial Andrew Wakefield paper on the link between MMR and autism.

The paper was set by the Assessment and Qualifications Alliance (AQA) exam board, who have since “apologised for any misunderstanding” and removed the exam from their website despite it being nearly 18 months old. I managed to track down both the paper and it’s accompanying mark scheme with a little Google sleuthing.

Question 5 is the relevant one. Part (a) asks pupils to explain how the MMR vaccine protects children from measles, mumps and rubella, whilst part (b) focuses specifically on Wakefield and his 1998 paper in The Lancet. Pupils must read the following passage and then answer some questions:

Autism is a brain disorder that can result in behavioural problems. In 1998, Dr Andrew Wakefield published a report in a medical journal. Dr Wakefield and his colleagues had carried out tests on 12 autistic children.

Dr Wakefield and his colleagues claimed to have found a possible link between the MMR vaccine and autism.

Dr Wakefield wrote that the parents of eight of the twelve children blamed the MMR vaccine for autism. He said that symptoms of autism had started within days of vaccination.

Some newspapers used parts of the report in scare stories about the MMR vaccine. As a result, many parents refused to have their children vaccinated.

Dr Wakefield’s research was being funded through solicitors for the twelve children. The lawyers wanted evidence to use against vaccine manufacturers.

The questions are:

(i) Was Dr Wakefield’s report based on reliable scientific evidence? Explain the reasons for your answer.

(ii) Might Dr Wakefield’s report have been biased? Give the reason for your answer.

For question (i) the mark scheme requires an answer of “no”, along with any two of the following: “sample size small / only 12″, “conclusion based on hearsay from parents”, “only 8 parents linked autism to MMR”, and “no control used”. The answer to question (ii) is given as “yes, being paid by parents / lawyers”.

The Daily Mail received the following response from Wakefield:

“The thought police appear to be saying, “To pass this exam you have to adopt this particular point of view.”

“We didn’t make any claims that MMR was the cause of anything. The exam question completely misrepresents what we said. The Lancet study received no funding whatsoever.”

Unfortunately for Wakefield, the lack of a link between MMR and autism is not just a “particular point of view”, but scientific consensus backed up by numerous studies contradicting his original in The Lancet. The exam question gives an accurate (if simplified) account of what happened.

This “controversy” over the exam is actually a complete fabrication by the Daily Mail. Their story tells us the Goverment has been accused of using the exam paper as indoctrination, but fails to mention who’s doing the accusing. It seems quite possible that the story’s author, Beezy Marsh, is also its subject. She is a well known opponent of MMR, as documented by Ben Goldacre.

It’s worth discussing though whether a question like this belongs on a GCSE Biology paper. Should pupils merely demonstrate that they know a bunch of scientific facts, or should they be awarded marks on their ability to understand scientific controversy?

I’d say the latter is an important part of the curriculum, and the Wakefield saga is definitely a suitable topic for the classroom. As a specific exam question however, I’m not so sure. The details behind the incident require an explanation more complicated than the few dotted lines provided by AQA allow.

It’s also incredibly cowardly of AQA to remove the paper from their website the moment “controversy” rears its head, which is why I’ve upload it to Just A Theory (scroll up for the link) for anyone to read. They’ve clearly thought about the response to the question though. An AQA report on the exam paper, which remains on their site for now at least, evaluates pupils answers.

In part (b) many candidates did not seem to appreciate the difference between bias and reliable evidence, often transposing the answers to (b)(i) and (b)(ii). In part (b)(i) many candidates offered ‘small sample’ and many others ‘reliance on parents’ opinion’, but only 10 % identified both ideas. In part (b)(ii) it was surprising that only half of the candidates recognised that payment by solicitors could lead to bias.

The fact that only half of the thousands of students taking the paper thought money changing hands might influence Wakefield’s decision shows that this is definitely a topic that should be covered in the curriculum. I’m just not sure that this particular exam question is appropriate. As for the Daily Mail’s accusation of “brainwashing”, perhaps a GCSE English retake is in order.

  1. 2 Comments

  2. I believe that you approach to this is correct. There is no merit in the Daily Mail’s case over MMR, but what I think is worse is that this puts ammunition in their hands and is very easily presented as a clumsy bit of government propaganda.

    The main problems with the question is that it really didn’t include any biology issues, nor did it set out much about how the trial was performed. I suppose we do know (by implication) that the trial was small and didn’t involve proper controls (only because the latter weren’t mentioned). However, that in itself doesn’t invalidate tests – some sort of tests showing strong biochemical causal links could still have produced compelling evidence. Barry Marshall produced some compelling evidence of the role of helicobacteria in stomach ulcers by a drastically simple trial. Of course we know that Dr. Wakefield’s MMR study had no such evidence, but you don’t know that from the question.

    Also the bit about some newspapers adopting it to run “scare stories” isn’t relevant to the biology at all. Of course it is very relevant in a media studies or social sciences course but it adds nothing at all to the question about the validity of the tests.

    Of course there is a general point about biases introduced by who finances studies, but that again is surely a social sciences type issue. Quite whether funding affects a trial or not depends on circumstances (although it often affects the way questions are framed).

    So absolutely include this as a curriculum item, and it would make a very interesting project for a class. But it is far too complex a subject to deal with from a few sentences. I’m still inclined to think that such things are dealt with better in social sciences than, say, biology. The issues of observer bias, objectivity and subjectivity, control groups and statistical analysis are far wider than just this subject.

    Incidentally, there are problems with other questions in that paper – the drugs one again looks to be treading into social sciences issues and the questions are mixing up matters of correlation and causation. If something should ever be on a curriculum somewhere it ought to be that issues – lots of policy makers and media outlets conflate the two frequently.

    On a general point, from somebody who didn’t take biology ‘O’ level, I’m sure I could comfortably pass this higher tier biology paper armed with nothing more than my physics, chemistry and maths knowledge, a smattering of general knowledge, what little powers of comprehension I have plus a bit of reading between the lines on what the examiner is leading to conclude. Frankly, I don’t think that is the way it should be.

    By steven Jones on Monday 11 May, 2009 at 5:54 pm

  3. nb. one other point (looking at the marking guidelines), the very last question over why some people object to genetic engineering asks pupils to suggest one reason why some people object to it. Interestingly the marking guidelines specifically disallows “religious reasons” or “unnatural” as valid answers. Of course these are not valid scientific objections, but they are most certainly those that many people give.

    So if the question said “name a risk of growing herbicide-resistant crops”, then that’s fair enough. However, if you ask the social sciences question that they did (in effect) “what objections do people have” then religious and “unnatural” are valid answers.

    By steven Jones on Monday 11 May, 2009 at 6:21 pm

Sorry, comments for this entry are closed at this time.