DfE informed of Ofqual’s algorithm flaws a fortnight before results

The Guardian reports that Education Secretary Gavin Williamson was informed of major defects in this year’s exam grading system a fortnight before A-level results for England were published.

The newspaper has been told that Williamson and the Department for Education (DfE) were then quickly given assurances by England’s exam watchdog, Ofqual, that any flaws could be rectified by permitting schools to appeal incidents of unusual grades.

Ofqual’s chiefs are to answer questions put by MPs from the Education Select Committee today (2nd September).

The details of prior warnings of the regulator’s flawed statistical methodology seen by the Guardian suggest that they will face a tough grilling.

Williamson himself may also face probing questions over his previous announcement that he had only become alerted to “real concerns” about A-level grades after they were officially released on 13th August.

The disclosures of a much earlier detection and notification of significant defects in Ofqual’s computer algorithm, which it used to nationally moderate school-based assessments, were provided by Cambridge Assessment, the operator of the OCR examination board (one of three such bodies administering GCSE and A-level qualifications in England).

Cambridge Assessment has revealed that it alerted ministers and the DfE in July – a full fortnight before the A-level results were publicly released.

It did so again a few days before the GCSE grades were released.

On both occasions, it had identified critical problems in how Ofqual’s statistical algorithm was allotting grades.

Remarks by Cambridge Assessment’s group director of research Tim Oates offer some support to Williamson, however.

He said that Williamson’s advisers and the school minister, Nick Gibb, were keen to hear of the issues that his organisation had identified.

He told the Guardian: “I cannot fault their reaction, the DfE responded swiftly and energetically, taking our analysis extremely seriously.”

His organisation’s internal research team detected peculiarities in the algorithm that penalised “outlier” students, such as gifted youngsters in bigger schools with below-average performance records and mid-ability pupils who had been assessed by teachers as deserving of C grades but who were instead bafflingly given fail grades (U) by Ofqual.

Oates revealed that once these irregularities had passed his organisation’s evidential threshold, the DfE was alerted immediately.

According to unnamed government sources, Ofqual had maintained to DfE representatives that the recently devised enhanced appeals procedure would be sufficient to manage what it insisted would be a minor number of irregular cases produced by its algorithm.

Williamson conceded in a recent statement to parliament that the appeals process, which Ofqual had maintained was fair and robust and capable of remedying any aberrant results from its algorithm, had proven inadequate.

There were, he said, “far too many inconsistent and unfair outcomes for A-level students” for even a significantly revamped appeals process to manage properly.

He added: “We took immediate action to provide certainty, as soon as it was clear that if we didn’t, too many students would receive grades that didn’t reflect their hard work and their ability.”

A timeline provided by Cambridge Assessment supports Williamson’s statement that he only became aware of serious flaws in Ofqual’s algorithm over the weekend after A-level results were released.

It confirms that the organisation alerted Ofqual’s chair Roger Taylor of algorithmic anomalies on 14th August, a day after the results were published.

The Profs Reviews

Close [X]