UK School Results awarded by Algorithm

[Silvanus]

One of the impacts of the pandemic here in the UK has been the cancellation of the vast majority of GCSE & A-Level exams in schools. For comparison with other countries' education systems: GCSEs are usually taken by pupils of ages 14 - 16, and A-Levels are usually taken around age 17-18. Vitally, A-Level results are usually one of the main criteria for university entry.

In the absence of the exams, the UK government requested that Ofqual (the UK's official exam regulator) develop a method to determine what the kids' grades will be this year. Ofqual have done so by developing an algorithm.

The algorithm calculates a student's final grade on several details:

* Firstly, the predicted grade given by the teachers and mock exam results. It's worth remembering that the vast majority of students improve between the mock exam and the actual exam, but this is not accounted for in the algorithm.

* Secondly, teachers were instructed to rank students' predicted performance from highest to lowest, and the pupil's ranking was also taken into consideration.

* Thirdly, and most controversially, "the most likely distribution of grades for each centre based on the previous performance of the centre". In short, the grades attained by past years were used to standardise and determine the grades for this year.

The "standardisation" caused by factors such as the test centre's past performance has had dramatic impacts on hundreds of thousands of grades. In total, 39% of grades were scaled down below their predicted grade, thousands of them by more than one grade: there are instances of predicted A and B being downgraded to D and C, as well as mid-level passing grades (C) being downgraded to outright fails (U).

As you may expect, protests have started around this, and the other political parties are demanding a U-Turn. Much of the criticism has focused on how this approach to standardisation entrenches inequality: the vast majority of downgraded results were from poorer schools, while wealthier schools got away relatively lightly, or had results scaled up above predicted grades instead. The approach seems to fly in the face of the government's supposed commitment to social mobility: in this case, it doesn't matter if a pupil works hard; if their school has performed badly in past years, their grade will be downgraded.

Most critics are opining that results should be based solely on teachers' predicted grades for the pupils. I imagine most of us can agree that Ofqual's approach is fatally flawed, but what would be the best approach? And is anything about the system they've implemented salvageable?

 

[Seanchaidh]

The best approach would be to suspend everything until you can have exams like before and provide the social support necessary to make that feasible. Which obviously won't happen.

edit:

I don't think this is the second best approach:

 

[Palindromemordnilap]

While I'm genuinely not sure what the best way of doing it would be, other than maybe finding a way to actually let these kids sit their exams, I'm fairly sure just setting an algorithm loose was probably one of the worst.

I also find Education Secretary Gavin Williamson's comments about using predicted grades "devaluing the results" to be somewhat tasteless since that is basically what this algorithm has gone and done

 

[Agema]

The approach seems to fly in the face of the government's supposed commitment to social mobility:

This government's idea of social mobility is evicting poor people from wealthy London boroughs to live in poor towns in the midlands and north, and ensuring houses in the countryside can be bought as holiday homes for the rich rather than primary residences for the locals.

 

[Silvanus]

The best approach would be to suspend everything until you can have exams like before and provide the social support necessary to make that feasible. Which obviously won't happen.

What impact would that have on university admissions? Defer all admissions for a year, or give students the option to use their predictive grades?

Suffice it to say my tongue was firmly in my cheek when I wrote that sentence.

 

[SupahEwok]

What impact would that have on university admissions? Defer all admissions for a year, or give students the option to use their predictive grades?

I'd honestly say that you guys should have your examinations. Leverage the private sector. In the US, our SAT and ACT college tests are administered in private testing centers or schools, and more professional exams are done so as well. Let the government pay such places for the additional space necessary to meet social distancing requirements, and stagger out test taking times to get everybody through them.

I mean, everything I've heard of your current government makes me doubt that such measures would ever be considered, but it doesn't seem like a problem without a solution.

 

[Agema]

The best approach would be to suspend everything until you can have exams like before and provide the social support necessary to make that feasible. Which obviously won't happen.

Much, much trickier than you might think. That also means propping up the entire HE sector for a year. Plus that the needs of the country means a year where it effectively doesn't graduate anyone would be extremely problematic by causing potentially devastating labour shortages in key areas.

* * *

What the best approach was, was for the government to not comprise lightweights who were selected for their loyalty to Brexit and libertarianism rather than their competence. Unsurprisingly, they've messed up just about everything they've touched, because they are to a man/woman almost uniformly useless. Thus with five months to think about and sort something out, it's been a balls up that's failed at the earliest prod.

Incidentally, was it an IT company owned by a friend of a government minister or that had worked for the Brexit campaign which got the contract to make up this algorithm?

 

[Specter Von Baren]

If I may use your own vernacular. Sounds like a bag of bollocks.

I feel like Ewok's idea sounds pretty alright, even if, as he said, the government likely won't take that route.

 

[Dwarvenhobble]

I'd honestly say that you guys should have your examinations. Leverage the private sector. In the US, our SAT and ACT college tests are administered in private testing centers or schools, and more professional exams are done so as well. Let the government pay such places for the additional space necessary to meet social distancing requirements, and stagger out test taking times to get everybody through them.

I mean, everything I've heard of your current government makes me doubt that such measures would ever be considered, but it doesn't seem like a problem without a solution.

Tests can't be staggered as everyone must in theory sit the same paper the same day to prevent any possible sharing of info on the content of the test. Often it can even be the invigilators aren't allowed to open the test paper packs until something like 30 minutes to 1 hour before the exams.

Space wise the UK has usually used school halls, sports halls and drama theatres as exam rooms and there's not much other things location wise near enough to schools to use. Plus going elsewhere means transporting the pupils and many schools at best has 1 minibus. Only in very rural areas are there school coaches etc.

Also before it's said no multiple papers would be a nightmare to do as each paper's marking must be standardised, meetings of exam boards held to determine what to do mark wise for issues like miss-spellings or other errors that can crop up. While there are multiple exam boards with their own exams each boards material and tests will be different so you can't just transfer people from one board onto another to do the tests.

 

[Satinavian]

I am not really sure how all of that works.


Sure, there is inequality between schools. There are schools were pupils tend to get better grades and others where they don't.

But wouldn't that also already be included in the "predicted grade" and mock exam ? If so, then applying this standardisation on top of it is not only unfair but inherently wrong. It is just magnifying all school differences by two and dishing it out to everyone.

 

[Dwarvenhobble]

I am not really sure how all of that works.


Sure, there is inequality between schools. There are schools were pupils tend to get better grades and others where they don't.

But wouldn't that also already be included in the "predicted grade" and mock exam ? If so, then applying this standardisation on top of it is not only unfair but inherently wrong. It is just magnifying all school differences by two and dishing it out to everyone.

It's difficult to say. I'm not fully on top of the new system (I no longer need to be) but the old system the mocks were often school administered so they would use older papers often the last years actual papers (as they'd generally not been released to the public) but not always so mock grades would be based on potentially different papers for different shcools.

Also mock exams generally take place right after Christmas or not long after with little to no study leave to revise in other than using the Christmas Holidays (really not fun)

With mocks

Some people won't even have finished all the course material so the mock grade is based on what they have done extrapolated to cover the stuff not looked over.

People may not have actually fully studied as hard as they would for the real thing due to the lack of study leave normally.

Also grades have been based on "Historic performance of the schools" and it's not been determined if that is by subject or overall.

So if you're at a school where there is a small elite top performers group and a large lower level to borrow an actual insult I've heard a student use "B-teckers" then it's quite likely the school normally sees lower grades thus under the new system the "B-teckers" would drag down the grades of the elites when under normal conditions they wouldn't impact one another at all.

Schools also all tend to use their own methods for determining predicted grades

 

[Agema]

I am not really sure how all of that works.

Sure, there is inequality between schools. There are schools were pupils tend to get better grades and others where they don't.

But wouldn't that also already be included in the "predicted grade" and mock exam ? If so, then applying this standardisation on top of it is not only unfair but inherently wrong. It is just magnifying all school differences by two and dishing it out to everyone.

Mock exams approximate to a load of bollocks - a huge number of students don't prepare properly or don't take them seriously and so underperform, often wildly so. I suppose they are useful in the sense that if a student does do well, it says something.

As someone who was an admissions tutor for two years, my view is that predicted grades are frequently unreliable: often overestimated as teachers attempt to support students to achieve their ambitions. In some cases I suspect the problem may be that students have been estimated high by the teachers and low by the algorithm, so there's a particularly huge disparity that's hard for the student to reconcile.

In reality, there is no "fair" solution. Every way of doing it is an estimate that will hurt at least some students. I kind of have a problem with the algorithm largely because it's a plainly bogus way of attempting to make the results look more "genuine" or "scientific", where more realistically just reorganising the way advantages and disadvantages were distributed.

 

[Satinavian]

There is no algorithm in the world that could correct teachers predictions. Teachers do have favourites and there are a lot of teachers who give good predictions all around while their colleages don't.

And that people don't prepare for mock exams as well as for the real ones is certain. But if an algorithm should include that, it would adjust estimates from mock exams towards better grades, not to worse.



The more i read about it, the more i am convinced a standardisation is not wrong but should be done on a country wide base so that this years studends have on average the same grades as past years but people from different regions or schools face still the same criteria.

 

[Silvanus]

I am not really sure how all of that works.


Sure, there is inequality between schools. There are schools were pupils tend to get better grades and others where they don't.

But wouldn't that also already be included in the "predicted grade" and mock exam ? If so, then applying this standardisation on top of it is not only unfair but inherently wrong. It is just magnifying all school differences by two and dishing it out to everyone.

The predicted grades are set by the teachers who've directly taught the pupil, and are supposed to take nothing into consideration outside that pupil's personal performance.

The algorithm "standardises" on top of that: it judges how well the school has done in past years, and then adjusts the present students' marks upwards or downwards towards that. If you're a well-performing student in a poor-performing school, your mark will be lowered; if you're a poor-performing student in a well-performing school, your mark will be raised.

Critics have been quick to point out that well-performing schools tend to be wealthier: they have better resources, may have a better chance of attracting staff, etc. So this algorithm is directly lowering the grades of clever pupils from poor economic backgrounds.

 

[Satinavian]

Critics have been quick to point out that well-performing schools tend to be wealthier: they have better resources, may have a better chance of attracting staff, etc. So this algorithm is directly lowering the grades of clever pupils from poor economic backgrounds.

Might even be worse. Better schools with better teachers, materials etc should not only achieve better grades in the real exams but also in the mock exams and (arguably) the teachers evaluation. So the better schools already give better estimates without this adjustment. Adjusting again on top of it would make the differences even more pronounced than on any year with real exams.

 

[Silvanus]

Might even be worse. Better schools with better teachers, materials etc should not only achieve better grades in the real exams but also in the mock exams and (arguably) the teachers evaluation. So the better schools already give better estimates without this adjustment. Adjusting again on top of it would make the differences even more pronounced than on any year with real exams.

It's hard to see this as unintentional. Where's the highest concentration of Conservative seats? Affluent south-east & south-west England.

 

[Dwarvenhobble]

The predicted grades are set by the teachers who've directly taught the pupil, and are supposed to take nothing into consideration outside that pupil's personal performance.

The algorithm "standardises" on top of that: it judges how well the school has done in past years, and then adjusts the present students' marks upwards or downwards towards that. If you're a well-performing student in a poor-performing school, your mark will be lowered; if you're a poor-performing student in a well-performing school, your mark will be raised.

Critics have been quick to point out that well-performing schools tend to be wealthier: they have better resources, may have a better chance of attracting staff, etc. So this algorithm is directly lowering the grades of clever pupils from poor economic backgrounds.

Worth pointing out the Scottish one literally was raising and lowering grades for pupils based on I think it was the house prices around the school lol.

Which luckily the English one isn't actually doing it's just doing it based on school previous performance which still has the issue you pointed out but maybe less so as you can have good schools in poor areas and vice versa. ( I won't say all the school manage those good results in an exactly ethical or fair way though)

 

[Burnhardt]

Space wise the UK has usually used school halls, sports halls and drama theatres as exam rooms and there's not much other things location wise near enough to schools to use. Plus going elsewhere means transporting the pupils and many schools at best has 1 minibus. Only in very rural areas are there school coaches etc

UK schools have been closed for months for most pupils. That means they have had plenty of empty space in the form of unused classrooms. There is absolutely no excuses, as to why exams could not be carried out, other than not getting enough invigilators/teachers in.

 

[Silvanus]

UK schools have been closed for months for most pupils. That means they have had plenty of empty space in the form of unused classrooms. There is absolutely no excuses, as to why exams could not be carried out, other than not getting enough invigilators/teachers in.

If you get hundreds of thousands of children into school at the same time across the country, it won't matter if the examinations themselves are socially distanced: there'll be pretty huge transference of the disease.

Plus, how fair will those exams truly be, if the children have missed out on months of teaching?

 

[Burnhardt]

If you get hundreds of thousands of children into school at the same time across the country, it won't matter if the examinations themselves are socially distanced: there'll be pretty huge transference of the disease.

Plus, how fair will those exams truly be, if the children have missed out on months of teaching?

That has not stopped students gathering together outside of school, for result day pictures, or en mass for the result protests.

Secondly, Teaching Unions have been claiming that teachers have still been working throughout the pandemic, so kids should not have missed anything, and should have the knowledge and preparation to be able to sit the exams just fine.

If the unions are lying, or some teachers have not been providing support to students, then students who are incapable of independent learning by A Level will get a very harsh wake-up call when they go to University, and will likely not graduate.

Finally, if examinations were never going to be an option, then courses should have been moved over to be entirely coursework based. That way final grades would be determined by actual work, and the students abilities, rather than predictions.