web analytics

Bill Bennett

Menu

Biased algorithm behind UK exam mess

There’s a nasty example of the damage poorly thought out algorithms can do from the UK.

New Zealand has an algorithm charter which could protect us from similar problems. Although that’s not certain, read on.

Thanks to the Covid-19 pandemic, schools in England and Wales were closed during this year’s exam season. The British school year ends in July and the main exams are held in June.

Students couldn’t sit exams the normal way. Instead the exam authorities set up an assessment system. Like other things these days, this meant going digital and using an algorithm.

The tyranny of a normal statistical spread

The exam regulators made a point of using a system that would give a normal statistical spread of grades. That way they could avoid grade inflation.

It’s important for another reason. In the UK there is stiff competition for the best university places. They go to the students with the best exam results. The entry conditions for certain courses can be strict and tough.

To get exam results, the regulators used an algorithm that combined grades given by teachers with a student’s past performance and the past performance of their school as a whole.

In many cases, as many as 40 percent of the total, the qualifications authorities marked students down, below the grades recommended by teachers.

Take from the poor, give to the rich

There was one huge problem with the exercise. It was skewed towards giving students from the ‘better’ schools a shift up and those from the underperforming schools a penalty.

In the UK the best schools are all in the richer areas. People pay a huge premium to buy a house in a better school zone. Which means the exam results rewarded students from better off families.

The bias was huge. The Guardian newspaper described the algorithms used as “a sociological sorting process which entrenches class divides in the state system”.

’…by building in a criterion of past school performance to this year’s A-level and GCSE results, Ofqual has tied the fortunes of individual students to pre-existing inequalities of outcome.”

Talent misses out

At first, many less well-off students who expected places at Oxford or Cambridge or, say, medical school missed out.

A-levels are important in the UK, to a degree they determine the next decade of a students’ life. They are more important than New Zealand’s NCEA exams in that sense.

This week the authorities backed down and went back to grading students based on teacher assessments. Which may fix matters, but after a huge amount of stress and upset plans.

New Zealand’s algorithm charter might not stop a similar abuse here but it could help. That’s because it makes algorithm decisions and the logic behind them transparent. The problem with the UK algorithm was less a lack of transparency and more a set of assumptions that are neither fair nor just.

«

»


8 thoughts on “Biased algorithm behind UK exam mess

  1. NZ used a somewhat similar system back in the 80’s/90’s when scaling Sixth Form Certificate results. One of my maths teachers showed me the curves that students were fitted to – it was basically impossible to fail if you studied Latin and very difficult to achieve a passing grade in Engineering (which was basically a metal shop class and attracted less academic students).

    The system used previous school performance as a guide to determine the following year’s marks, which had some interesting corner cases: I switched from German to Engineering in the 6th form, so my (quite high) School Certificate German mark came across and was given to somebody deserving in my Engineering class. The rest of the class got mostly failing grades despite having completed the required work; my mark ended up as an 8 on the 1-9 scale.

    https://en.wikipedia.org/wiki/Sixth_Form_Certificate

  2. The problem with the UK algorithm was less a lack of transparency and more a set of assumptions that are neither fair nor just.

    The Guardian newspaper described the algorithms used as “a sociological sorting process which entrenches class divides in the state system”.

    Working as designed then. And, yes, I’m pretty sure that it was designed that way so as to ensure that students from good families didn’t have to rough it with everyone else.

Reposts

Leave a Reply

Your email address will not be published. Required fields are marked *

%d bloggers like this: