UK School Results awarded by Algorithm

[Silvanus]

That has not stopped students gathering together outside of school, for result day pictures, or en mass for the result protests.

Well sure, but there's quite a difference between here-and-there gatherings for results day or protests, and almost every student in the country.

Secondly, Teaching Unions have been claiming that teachers have still been working throughout the pandemic, so kids should not have missed anything, and should have the knowledge and preparation to be able to sit the exams just fine.

Is remote learning really going to be fully comparable to in-classroom learning?

If the unions are lying, or some teachers have not been providing support to students, then students who are incapable of independent learning by A Level will get a very harsh wake-up call when they go to University, and will likely not graduate.

A-Levels and GCSEs really aren't that much about fully independent learning: they're supposed to be highly structured. Pupils analyse the sources they're given, under direction; they don't do much in the way of independent research. It's certainly not comparable to university, which is why most the marks for the first year of many Uni courses don't even count towards the final grade.

 

[Agema]

Finally, if examinations were never going to be an option, then courses should have been moved over to be entirely coursework based. That way final grades would be determined by actual work, and the students abilities, rather than predictions.

Speaking as someone who has spent five months doing lots of work to redesign teaching delivery and assessment in a covid-19 world, I'd point out that this is fraught with complications and easier said than done, especially when the assessment has to be co-ordinated on a scale as large as national.

 

[Specter Von Baren]

Speaking as someone who has spent five months doing lots of work to redesign teaching delivery and assessment in a covid-19 world, I'd point out that this is fraught with complications and easier said than done, especially when the assessment has to be co-ordinated on a scale as large as national.

How's that going? Is your job doing ok with how things are now?

 

[Agema]

How's that going? Is your job doing ok with how things are now?

I am, broadly, extremely hard to make redundant. I don't mean that in the sense that I'm amazing at my job or irreplaceable, just that I fill necessary teaching and administrative roles on a degree (medicine) that must continue even in the face of apocalypse. If the roles must exist and I do them okay, I'm secure.

Bluntly, HE degrees this year were a fudge. They're nothing like as bad as the A-levels have been because the majority of the assessment in many courses was already done (coursework or January exams), and higher education was more "nimble" to make adjustments than secondary, but even still May exams were rapidly converted in ways that will have not been good for their reliability and integrity, because there just wasn't time to do a proper job on it. Most universities have systems which prevent student disadvantage (otherwise universities get sued), which means in almost all cases the error is going to be positive, so students grades will be higher than normal.

Teaching has been largely moved online. This has mostly been simple, but where it has not been simple it has been very difficult: finding appropriate software, testing it to ensure it meets needs, new teaching techniques. Thankfully, for various reasons, a lot of universities including ours already had some knowledge and experience of online teaching systems, so we managed to convert rapidly to finish the last academic year okay, and we've been able to build on that for 20-21. Some things (laboratory work, clinical skills) however must be taught face to face, so there's been a massive faff sorting that out so it can be done within safety guidelines, whilst also meeting standards for accrediting bodies; this also means complex timetable reorganisations and so on. My teaching is actually straightforward - it's my admin role that's dragged me into a lot of that nightmare. In terms of assessment, coursework is mostly not an issue. End of year exams should be back to normal, however semester 1 exams probably won't (including some of mine), so these have to be redesigned to be done properly rather than the half-way house fudge of May 2020, which I plan to have done by end of this month. And then there are the 20,000 emails from students to reply to about what we're doing, why we're doing it, demanding fee reductions, permission for late arrival absence, being excused from A, B and C, threatening to sue us, etc.

Yeah. I've actually spent a fair chunk of my "annual leave" this year working to sort plenty of this stuff out, and I won't be getting any time in lieu back.

 

[JoJo]

Unsurprisingly, the government has done a u-turn:

A-levels and GCSEs: U-turn as teacher estimates to be used for exam results

Education Secretary Gavin Williamson apologises to students and parents for the "distress" caused.

www.bbc.co.uk

It was probably never viable for England to be adhering to a harsher system than the other three UK nations. My younger sister, who was scheduled to take her GCSEs this year, will be relieved.

 

[Agema]

Unsurprisingly, the government has done a u-turn:

What else do you expect from a bunch of dilettantes who understand nothing but public opinion polls?

Of course, again speaking from HE, this is utter chaos for admissions.

 

[Specter Von Baren]

I am, broadly, extremely hard to make redundant. I don't mean that in the sense that I'm amazing at my job or irreplaceable, just that I fill necessary teaching and administrative roles on a degree (medicine) that must continue even in the face of apocalypse. If the roles must exist and I do them okay, I'm secure.

Bluntly, HE degrees this year were a fudge. They're nothing like as bad as the A-levels have been because the majority of the assessment in many courses was already done (coursework or January exams), and higher education was more "nimble" to make adjustments than secondary, but even still May exams were rapidly converted in ways that will have not been good for their reliability and integrity, because there just wasn't time to do a proper job on it. Most universities have systems which prevent student disadvantage (otherwise universities get sued), which means in almost all cases the error is going to be positive, so students grades will be higher than normal.

Teaching has been largely moved online. This has mostly been simple, but where it has not been simple it has been very difficult: finding appropriate software, testing it to ensure it meets needs, new teaching techniques. Thankfully, for various reasons, a lot of universities including ours already had some knowledge and experience of online teaching systems, so we managed to convert rapidly to finish the last academic year okay, and we've been able to build on that for 20-21. Some things (laboratory work, clinical skills) however must be taught face to face, so there's been a massive faff sorting that out so it can be done within safety guidelines, whilst also meeting standards for accrediting bodies; this also means complex timetable reorganisations and so on. My teaching is actually straightforward - it's my admin role that's dragged me into a lot of that nightmare. In terms of assessment, coursework is mostly not an issue. End of year exams should be back to normal, however semester 1 exams probably won't (including some of mine), so these have to be redesigned to be done properly rather than the half-way house fudge of May 2020, which I plan to have done by end of this month. And then there are the 20,000 emails from students to reply to about what we're doing, why we're doing it, demanding fee reductions, permission for late arrival absence, being excused from A, B and C, threatening to sue us, etc.

Yeah. I've actually spent a fair chunk of my "annual leave" this year working to sort plenty of this stuff out, and I won't be getting any time in lieu back.

Glad to know you're at least financially secure and this also answers my unvoiced question of why you hob knob with us. If you're having to deal with this much bureaucracy all the time then these forums must be refreshing.

 

[Dwarvenhobble]

UK schools have been closed for months for most pupils. That means they have had plenty of empty space in the form of unused classrooms. There is absolutely no excuses, as to why exams could not be carried out, other than not getting enough invigilators/teachers in.

They been closed for all but essential workers children. Not fully closed technically.

Also while you could argue about doing the tests in classrooms there are fairly strict rules about posters etc which could be seen to give pupils additional information or an advantage. Plus you'd need board invigilators in each room or area you use.

 

[Dwarvenhobble]

What else do you expect from a bunch of dilettantes who understand nothing but public opinion polls?

Of course, again speaking from HE, this is utter chaos for admissions.

I don't get the looking down on governments for doing the right thing. Yes they U turned because it very much needed to happen.

I do wonder why HE institutions didn't say "We won't be accepting people for 1 week or doing clearing etc as a precaution this year"

 

[Agema]

I don't get the looking down on governments for doing the right thing. Yes they U turned because it very much needed to happen.

The problem is, why did they make such a mistake in the first place? How did they expect to tell a student expecting AAB that they have BDU because an algorithm said so? How did they not realise how badly this was obviously going to go wrong? The best answer for a prospective student that's had their plans shattered is to spend a year doing a low paid job (if they can even get one, coronavirus recession and all) so they can take their A-levels for 2021 admission. It's pretty cavalier to waste a year of someone's life that way, isn't it? Never mind the shock, anxiety and so on.

I do wonder why HE institutions didn't say "We won't be accepting people for 1 week or doing clearing etc as a precaution this year"

Basically, because they can't: due to a host of rules, regulations and agreements set with government, and for a host of practical reasons that there's a huge amount universities and students need to organise in the short window between grade release and start of the academic year. UK universities, incidentally, are kind of weird in the sense that they are de jure private sector, but de facto public sector, as the government controls and regulates the seven shades of shit out of them. I might cynically note that the government's desire to control the sector is inversely proportion to its willingness to pay for it.

Anyway, fundamentally, it's the job of the government to let everyone know what schoolkids have got in their exams on the official date they said they would.

Glad to know you're at least financially secure

Thank you. I hope you're doing well too.

If you're having to deal with this much bureaucracy all the time then these forums must be refreshing.

It's a way to blow off steam and I enjoy talking about stuff. But you can probably tell how good/bad a day I'm having by how light-hearted or snarky my posts are.

 

[Specter Von Baren]

Thank you. I hope you're doing well too.

It's a way to blow off steam and I enjoy talking about stuff. But you can probably tell how good/bad a day I'm having by how light-hearted or snarky my posts are.

The owners of the place I work at are very stubborn, they won't let this pandemic sink their business without a fight. And I'm doing good, hopefully I'll be able to pay off my car in the next few months, so I'm looking forward to that.

And yeah, we all have our bad days, the important thing is that none of us hold a grudge over things and can have fun still.

 

[Dwarvenhobble]

The problem is, why did they make such a mistake in the first place? How did they expect to tell a student expecting AAB that they have BDU because an algorithm said so? How did they not realise how badly this was obviously going to go wrong? The best answer for a prospective student that's had their plans shattered is to spend a year doing a low paid job (if they can even get one, coronavirus recession and all) so they can take their A-levels for 2021 admission. It's pretty cavalier to waste a year of someone's life that way, isn't it? Never mind the shock, anxiety and so on.

It depends on the algorithm honestly.They thought their version would be better than the Scottish one. It turned out to have problems almost as bad though.

On the subject of taking a year off........ honestly with Coronavirus and everything going on if they can I'd say take the year break because Uni will lost a lot of the social aspect until at least next year and yes it's a year of life but it's a year to learn the world and appreciate the worth of a degree.

I wouldn't say delaying results would be good or delaying and having to have two different year groups doing A-level tests in 2021 would be good idea though honestly

 

[Trunkage]

Cant speak to solution for the UK.

But schools have been open for most of the time during the pandemic in Oz. We had 6 weeks of home schooling with vastly different results from even teachers with our school, let alone across schools.

Also, somehow, my daughter has gone from mainly B to sometime C student to flat out D all around in 6mths. Whatever is happening, I dont think its working. And, quite possibly, it's teacher dependent

 

[Agema]

It depends on the algorithm honestly.They thought their version would be better than the Scottish one. It turned out to have problems almost as bad though.

I've actually done this sort of thing myself, because sometimes we need to forms of conversion of scores into marks, and we likewise have algorithms to do this. But we don't just pick an algorithm and let the numbers crunch, we also do "reality checks" on the output to make sure they are reasonable. If no-one checked the output, it's a horrible oversight. (I'd actually think for something of this magnitude, it's quite possible they'd have an independent, external analyst to scrutinise.) We all know the sorts of things they analyse when grades come out every year, which include different types of schools, variability by location, SES, etc. so a skilled analyst would surely have checked these as well.

Whether they didn't know what this algorithm would do, or they did know and did it anyway, either way I think they deserve a healthy dose of criticism.

On the subject of taking a year off........ honestly with Coronavirus and everything going on if they can I'd say take the year break because Uni will lost a lot of the social aspect until at least next year and yes it's a year of life but it's a year to learn the world and appreciate the worth of a degree.

Maybe. But like I said, it's a year where it may be really hard to get a job, so little money or useful work experience, and with social life disrupted - never mind that for many their friends will have gone to uni, meaning they'll be kicking their heels in their home town.

I wouldn't say delaying results would be good or delaying and having to have two different year groups doing A-level tests in 2021 would be good idea though honestly

There are optional retakes in October this year for those who feel they will do better that way, so as long as they get the results before mid-January they should have a small advantage, which is that for the majority of courses they can have the grades in hand and get an unconditional offer.

 

[Silvanus]

Glad to see they've U-Turned (and so shortly after Gavin Williamson insisted in a live interview that there would be no change). From the start, from the optics alone, I thought it was nearly inevitable.

If no-one checked the output, it's a horrible oversight. (I'd actually think for something of this magnitude, it's quite possible they'd have an independent, external analyst to scrutinise.)

I haven't seen anything about testing the 2020 output, but according to the New Statesman, they did test the algorithm by applying it to 2019;

When Ofqual tested the model, it didn’t examine the potential pitfalls of the ranking measure. Instead, it tested for accuracy by using data from the years 2016, 2017 and 2018 to predict the grades of 2019. But it used the actual rankings of the students in the 2019 exams to test for the model’s accuracy. By contrast, it relied on teacher’s estimates to predict scores in the 2020 exams.

Even without this consideration, testing found the accuracy of the model to be fairly low – in the range of 50 to 60 per cent. “One in three might be misclassified anyway, if not higher, and that’s when you use the correct rankings in the actual exams in 2019,” says Wilkinson. “With teacher predicted rankings, that could actually go down significantly.”

This is utterly incredible incompetence. They were happy to go along with it after their test showed up to half of all pupils being inaccurately marked?

 

[Silvanus]

Interesting article from a higher education site: https://wonkhe.com/wonk-corner/they-met-the-terms-of-their-offer-arent-we-required-to-admit-them/

In short, UK courts have already recognised the right of students to be admitted to a university that offers them a conditional offer (when they successfully meet the grade the university asks). Technically its a contractual obligation.

So, now we have a situation in which students 1) Got an offer from a university, conditional on them getting a certain grade (which is common practice); 2) Failed to reach the grade after being downgraded by the algorithm; 3) Successfully reached the grade now that it's based on predictive grades instead.... only to find the university has now given away all its spaces.

It was always going to be a difficult situation, but... bloody hell, what an enormous, unnecessary mess.

 

[Agema]

In short, UK courts have already recognised the right of students to be admitted to a university that offers them a conditional offer (when they successfully meet the grade the university asks). Technically its a contractual obligation.

So, now we have a situation in which students 1) Got an offer from a university, conditional on them getting a certain grade (which is common practice); 2) Failed to reach the grade after being downgraded by the algorithm; 3) Successfully reached the grade now that it's based on predictive grades instead.... only to find the university has now given away all its spaces.

It was always going to be a difficult situation, but... bloody hell, what an enormous, unnecessary mess.

Yep, basically. Total screw-up.

The government's accusing Ofqual, I see. I've noticed that since 2010, the government has increasingly created a scapegoating tactic, where it creates layers of semi-autonomous officials and organisations to do things which seem to me to be mostly designed to insulate ministers from blame when there are screw-ups.

 

[Satinavian]

The more i read about it, the more i am glad that Germany held proper exams this year and we avoided all those issues.

 

[Buyetyen]

The 21st century is rapidly showing us that whatever potential algorithms have as a tool is almost irretrievably squandered by people who think that the algorithm can automate their job so they get to collect a paycheck just for showing up.