Students in deprived areas of Scotland were more likely to have their grades pulled down by the exams regulator than those in well-off parts of the country, it emerged this week.
Some otherwise high-achieving students complained of being marked down in all-important Highers exams – not because of their own ability or attainment, but because their school had performed badly in the past.
Schools in deprived areas tend to have worse exam records on average. So the decision to use an institution’s historical records in the allocation of grades has affected pupils in poorer parts of Scotland more than their richer peers.
One student said she was “devastated and really nervous for the future now” after the A and two Bs she had been predicted were brought down to three Cs and a D, jeopardising her place at Stirling University.
And it looks like exam watchdogs in the rest of the UK are on track to replicate the Scottish controversies when they release GCSE and A-level grades later this month. The English, Welsh and Northern Irish regulators all plan to take account of a school’s past performance when adjusting student’s grades this year.
What happened in Scotland?
Pupils throughout the UK have been unable to sit their usual exams because of the pandemic, leaving regulators with tough choices to make about how to award qualifications.
The Scottish Qualifications Authority (SQA) asked schools to estimate what students would have got if teaching and exams had gone ahead as usual. It then moderated those results in an attempt to even out differences between schools and between years.
As it turns out, the SQA felt that many teachers had been too generous with their estimates. Of the half-a-million individual grades submitted by schools, 124,564 were adjusted down. Fewer than 10,000 were bumped up. Nearly all the adjusted results (96 per cent) were changed by a single grade.
One of the factors that the SQA used to decide whether a grade should be changed is the school’s performance over the last four years.
For some individual students, this was very significant. Previously high-achieving pupils who were on track for top marks had their grades reduced to reflect their school’s past performance, rather than their own work and abilities.
And it had a disproportionate effect on students in economically disadvantaged areas. This seems to be because the poorer the area, the more likely the school is to have a bad exam record.
In the most deprived areas of Scotland, SQA moderators reduced the proportion of students getting A-C grades in their Highers by 15 percentage points compared to teacher estimates. In the richest parts of the country, they were brought down by 7 points.
Asked about this disparity at a recent press conference, Scotland’s First Minister Nicola Sturgeon told Channel 4 News the new moderation system was “necessary to make sure we have a credible – and that’s important for young people – system of results”.
She added: “Without that system of moderation, I would be saying that 85 per cent of young people in our most deprived areas had passed Highers this year, compared to around 65 per cent last year and in previous years.”
“Poorer young people don’t do as well as more affluent young people and that is something we are working very, very hard, from the early years right through our school system, to try to rectify.”
She said students in Scotland would be able to appeal their grades.
What’s happening in the rest of the UK?
It looks like regulators in England, Wales and Northern Ireland are on course to replicate these controversies when GCSE and A-level grades are released this month.
Ofqual, the English exams watchdog, has its own rules for moderating grades. We won’t know the precise details until results day, but it seems they’ve adopted a similar moderation process to Scotland, which could see pupils downgraded simply because of where they go to school.
Ofqual guidance says their model will “consider each centre [school or college] individually, comparing centre assessment grades to the centre’s historical results taking into account the prior attainment of the current students, to judge whether its centre assessment grades are more generous or severe than predicted”.
In other words, they’ll be adjusting “centre assessment grades” (teachers’ estimates of what their students should have got) by looking at the school’s past performance. That’s what proved to be the sticking point in Scotland.
And Ofqual seems to explicitly rule out the use of data on individual students: “The process will consider prior attainment at centre level, not at individual student level; students’ individual performance will not be predetermined by their prior attainment at Key Stage 2 or GCSE.”
So it’s possible that, as in Scotland, students who achieved top marks throughout their academic careers will have their A-level grades reduced if their school has historically performed badly.
This concern was raised with Ofqual in the spring during its public consultation on how to moderate school-estimated grades this year. In the summary of responses, the regulator quotes a teacher as saying: “I am worried that individuals who would have excelled given the chance to sit exams are going to be unfairly penalised due to poor performance of their predecessors.”
And the effects won’t just be felt by individual students. So-called “turnaround schools” – institutions that have recently improved their exam results – risk being penalised today for past failings.
Responding to the Qfqual consultation, a school or college said: “For schools which have struggled in the past and are predicting a steep improvement in results, it would be devastating to use historical results.”
But Ofqual says: “While we recognise that a small number of centres would like standardised grades to reflect their recent or expected improvements in results, our research into GCSE grading shows the performance of centres rarely improves (or deteriorates) consistently in the short term. On balance, it is therefore best not to try to predict improvements in performance.”
An Ofqual spokesperson told FactCheck: “Although exams have been cancelled because of coronavirus, most students will still be able to move on to further study or employment as planned, with calculated grades – based on judgements from their schools or colleges and standardised by exam boards to make sure they’re consistent between centres.”
The regulator confirmed last night that schools in England will be allowed to appeal grades if they can show that historic data does not represent this year’s cohort. But unlike in Scotland, individual pupils will not be able to challenge their marks.
Meanwhile the Welsh exams watchdog, Qualifications Wales, will also be using “historic performance, adjusted for the ability of this year’s cohort” to work out if teacher’s estimates should be changed.
A spokesperson told FactCheck: “The approach to awarding this year’s grades has been carefully thought through to be as fair as possible in the circumstances and to protect the value of results. The standardisation process will use information about an exams centre’s historic performance, but this will be adjusted to reflect this year’s students so that learners across Wales are treated in the fairest possible way.”
Qualifications Wales says that the teacher-estimated grades were “generous”. Without standardisation, 40 per cent of A-levels would have received a grade A* or A, compared to 27 per cent in 2019. Such an increase “would not be credible”, according to the watchdog’s chief executive, Philip Blaker.
And in Northern Ireland, the regulator will also use a school’s results in the previous three years as part of the “statistical standardisation model” to work out if teachers are being too harsh or too generous.
A spokesperson for the Northern Irish CCEA told FactCheck: “In the absence of the summer 2020 examination series, CCEA was directed by the Minister for Education to award grades to learners that would allow them to progress to further study or employment. An important element of the instruction was that qualification standards were maintained and the distribution of grades would follow a similar profile to that in previous years. We were required to use teacher professional judgment with statistical standardisation to achieve this.”
They also told us: “if centre assessment grades were not standardised, we would see results for 2020 that were significantly higher than have ever been seen before. Improvement on such a scale in a single year would significantly undermine the value of the qualifications that the student will receive. Therefore standardisation was essential to ensure the value of qualifications were maintained.”
We won’t know how much weight the regulators will put on historic records until the results are published. But the fact that all three intend to use this data when moderating this year’s students means it’s entirely possible that we’ll see the controversies in Scotland replicated throughout the UK this month.