The claim
“In four of the last five years, there have been no mistakes made in the setting of school examination papers. Since the 16th of May this year, there have been ten such mistakes made. What does the Prime Minister intend to do for those amongst the 250,000 young people affected who lose either their university of choice, or their university at all, because of this staggering incompetence?”
John Mann MP, 22 June, 2011
The background
Thousands of secondary school pupils have had the already-nailbiting exam period made even more stressful thanks to a string of errors in exam papers that made some questions impossible to answer.
John Mann, the Labour member for Bassetlaw, went on the attack in Parliament today, quoting some figures that suggest the sudden flurry of mistakes does indeed represent a dark time for the boards that set examination papers.
But have Labour been swotting up sufficiently to take the Prime Minister to task on this issue?
The analysis
FactCheck contacted Ofqual, the Government-funded, independent body charged with regulating qualifications, exams and assessments in England.
They have released figures which suggest that, while the situation does appear to have got dramatically worse this year, Labour haven’t got the numbers right.
In a statement, Ofqual’s chief executive Glenys Stacey said a number of live exam paper errors have in fact been reported to the regulators over the last five years.
One incident concerning a GCSE paper was reported in 2006 and another mistake on an A-level paper was flagged up in 2007. In 2008 there were three incidents, involving two GCSE and one A level paper, while there were no reports of errors in 2009 and 2010.
So that’s a few immediate lost marks for Mr Mann on the first part of his claim, and Ofqual, which was only created in 2010, also adds an important qualification.
“Our reporting arrangements will be different to those in place in the past and so we cannot state for certain that these figures below capture every incident that occurred.
“The figures should therefore be treated with caution, and not used to draw direct comparisons with this year’s figures.”
So unless Labour has found a foolproof set of statistics the current exam regulators don’t know about, it appears to be difficult to talk about a sudden explosion in mistakes with absolute certainty.
Mr Mann scores better marks when he says that there have been ten mistakes made since May 16 this year, though in the interests of accuracy we should say that total appears to be slightly inflated.
As far as mistakes on the exam papers handed out to students, there have been nine reported so far out of 5,000 different GCSE, AS and A levels, according to Ofqual.
The largest English exam board AQA have made errors in three AS papers (in business studies, geography and computing), as well as a GCSE maths exam where some papers had questions from a previous paper included.
Fellow setters OCR sent out a GCSE Latin paper which contained three errors, an A level physics paper which used the wrong measurements and a faulty maths AS paper.
There were also complaints about a biology AS level paper set by Edexcel and a CCEA business studies GCSE paper.
Ofqual said one of those nine mistakes was a minor printing error. They also received a “serious complaint” about one of the marking schemes given to teachers assessing their students’ coursework, which may have led to the reports of ten cases in total.
More importantly, how many students have really been affected and how will the mistakes alter their chances of getting into university?
Well, Ofqual take issue with Mr Mann’s claim about “250,000 young people affected” and put the real number at 100,000.
They also dispute that students will have the prospect of getting into their university of choice ruined by having the bad luck to have been handed a flawed exam paper.
Ofqual said: “Students who have sat one of the question papers that included a mistake can be assured that the awarding organisations have procedures in place to make sure that so far as possible, no candidate is disadvantaged.”
The exam boards have gone further, with AQA saying: “We will ensure that no student loses out as a result.”
The body, which is a registered charity, said that, depending on the situation, it would either give all students full marks for the problem question or not include it in the marking.
AQA also said markers would “take into account any impact on the performance of all students taking the paper”, adding: “In addition, where a student may have been adversely affected in answering subsequent questions, schools and colleges can apply for special consideration.”
All the exam boards have promised to introduce added checks to make sure the problems are not repeated next year.
The verdict
David Cameron said he met the Education Secretary, Michael Gove, today, and has been assured that Ofqual “are taking the toughest possible action to root out this failure and make sure it doesn’t happen again”.
It may prove to be virtually impossible to root out human error in such a huge undertaking as setting and marking all the exams sat by British pupils.
And the scale of that endeavour does have to be borne in mind when considering the problem. Although a rise from an average of one mistake a year to nine does look like “staggering incompetence”, that’s nine out of 5,000 different GCSE, AS and A level papers, an insignificant difference to a statistician.
Of course, that will be small comfort to those 100,000 or so pupils – ten per cent of the total number who sit GCSEs, AS and A levels each year – who have had the strain of exam time compounded by worries about whether the mistakes in the papers they sat will hurt their future prospects.
Only time will tell whether the exam boards really manage to avoid disadvantaging those young people – but Labour certainly won’t help to soothe their anxieties by exaggerating the problem.
By Patrick Worrall
