What is statistical standardisation anyway?

On 29 June, we published decisions on arrangements for summer 2020. We confirmed the aims that will underpin the statistical standardisation model to be used by WJEC to calculate grades for GCSEs, AS, A levels and the Skills Challenge Certificate.

By Tom Anderson, Head of Research and Statistics, Qualifications Wales

Many of those who took part in our consultation raised excellent points and questions about standardisation. But some people said they felt unclear on what standardisation is.

We want to explain standardisation as clearly as possible so that learners and parents can understand how grades will be arrived at.

So, what is statistical standardisation?

There is a lot of detail about the standardisation models that will eventually need to be explained. But, as a high-level introduction, standardisation will calculate grades that are, overall, broadly similar to grades awarded in other years. The approach will use the best combination of evidence available in these extraordinary circumstances.

In June, each centre provided a centre assessment grade and a rank order to WJEC for all learners studying GCSEs, AS, A levels and the Skills Challenge Certificate. Following on from that process, there are two main steps to standardisation:

  • Step 1: Calculate grades for learners in each centre. WJEC will calculate a set of grades for each exam centre, such as a school
  • Step 2: Allocate grades to individual learners. WJEC will allocate the set of grades from step 1 to individual learners using the rank order provided by the teachers in the centre

It is the combination of the evidence in these two steps which is the strength of the model.

Different methods for standardisation have been considered in detail by exam boards and qualifications regulators across the UK.

Data from recent years, such as prior attainment data and historical centre data, is being used to understand which method is likely to produce the most accurate set of grades in step 1. Accuracy will be a primary consideration for the selection of the final approach in Wales.

Although the standardisation methods being considered across jurisdictions are similar, there will be some differences in the models used by WJEC in Wales compared to other UK jurisdictions.

This is because differences in education systems and qualification designs mean that the data available to calculate grades in step 1 varies. For example, AS grades are still linked to A level grades in Wales and Northern Ireland, whereas in England an AS grade does not contribute to an A level grade.

Research to date suggests that using AS performance to calculate the A level grades in step 1 of standardisation will improve the overall accuracy of the grades, compared to not using that evidence. This is because the AS units contribute 40 per cent to the final A level grade and because most learners taking A levels have already been awarded AS units. Evidence from AS performance will therefore be used to standardise grades at a centre level in Wales.

The same considerations apply to some GCSEs, such as Physics, Biology and Chemistry, where most of the learners entered this summer have already been awarded a unit which forms 45% of the overall grade.

For AS and most GCSEs, research suggests that alternative data can be used in step 1 of the model. This would be the recent centre performance and the previous attainment of this summer’s learners in the centre.

For AS, the previous attainment of this summer’s learners in the centre would be based on GCSE performance, which can be used to work out if AS results in the centre should be higher or lower this year.

Whilst for GCSEs, research is being completed into how learner performance in key stage 3 teacher assessments and tests taken in year 9 can be used to make the same sort of change to recent centre performance.

Although the use of that data for calculating GCSE grades may be controversial to some, it is also the only consistent evidence available that could be used to take account of how strong learners are this year compared to previous learners in the centre.  Research completed to date shows that this data is likely to improve the accuracy of the grades calculated this summer.

In summary, once the grades are allocated to learners using the rank order provided by their teachers, learners should receive a fair set of grades across their qualifications that enable them to move on to the next stage of their lives. Once grades are counted up across Wales, results should be broadly similar to previous years, protecting the credibility of those grades and fairness to learners across years.

WJEC will make final proposals to us, as the regulator, for the standardisation models. We will approve the final models to be used by WJEC.

We are working on providing more specific information on the final models to be used in Wales for GCSE, AS, A level and the Skills Challenge Certificate later in July, with more detail to follow once learners have received their grades in August.

Published 14 July 2020