Review Scores are Shit

[Stewie Plisken]

That makes sense to someone who reads a lot of reviews, but what is the baseline of "average" for someone who does not play a lot of games, or has no reference of best and worst parts of a genre (racing games for example). It is not like review sites have guides on how to read their scores, and how they are different to other site's ratings. I mostly agree with how you placed the numbers, but from what I see, IGN considers 7 to be average, and Polygon places its average at 5, and that's not even mentioning the quality of the written review can be very different as well.

A scale of 5 with clear distinctions is the one I find more agreeable myself: 1= broken game (nigh-unplayable), 2= bad (skip it), 3= average (in the sense that it can be skipped, but it's not bad), 4= good (buy it), 5= excellent (really, really buy it, kill your neighbor and steal their copy if necessary). Naturally, in this scenario, 1 and 5 are used rarely, if ever.

Of course even that's just a compromise for me. Scores are indeed bullshit, particularly because they do more harm than good. They encourage a culture that instantly looks at a score and skips the extended review, where the critic elaborates his/her thoughts. This is potentially good for heavy-hitters in the AAA industry, but it's bad for the consumer and the smaller studios, even if at first glance it looks helpful. That's not even accounting for shit like Metacritic score affecting bonuses for the devs.I guess that's why so many do a 10-scale or add the . (.5) on the 5-scale; you need to spread the scale out to differentiate a good game from a really good game that's not excellent. But that's also the point where subjective opinion and experience get into the equation and it feels more like a cop-out for the reviewer in that they don't need to carry the burden of a recommendation as heavily as they would otherwise.

Alternatively, a Yay or Nay is probably the best way to go about it. The reader will probably have to at least glance at the review for a little more information and the recommendation stands loud and clear.

 

[EricMcArthur]

I'm personally not on board with this Pollyanna-ish quest to rid the gaming world of review scores. It didn't take an industry-wide mind-meld for critics, as a whole, to tell you that Duke Nukem Forever sucked, or that Uncharted 4 is great. Some critics value certain elements that other critics find less important. But that's the point. You get enough critics reviewing the same piece of art, or the same product, and you get a general "critical consensus". I personally think calling every game with 69 and under "weak" is ridiculous - and that new aggregator that The Escapist seems to be in bed with has a lot to learn there. That 50 - 74 range is much more properly called mixed/average as Game Rankings does. Bottom line - this is not science. Go with your gut when grading a game. Think about it like the thought-process you have when your buddy asks you "What did you think?" when leaving the latest X-Men movie. Do you balk at that and say, "No, I'm repulsed by such reductive criticism. You'll have to wait for my full review on Rotten Tomatoes"? Sure, grading is hard. Ever try to grade an AP English essay? Or an SAT writing sample? IT's brutal. And it has real-world effects. But it has to be done. I'm hearing that people want to get rid of the GPA, and testing in general God help us....

 

[Saelune]

I don't think scores are bad, its just how they are used. A 3/5 means nothing. A 7/10 shouldn't be considered a bad score.
And people should realize not liking a game doesn't mean its bad.

I think it gets silly when it tries to get super specific with scores. I like a clear 10 point scale. No .5's though.

 

[Tanis]

I like reviews that break down the elements.

Music, Game Play, Story, Graphics, Voice Acting, Glitches/Bugs, Etc.

I'll play a game with mediocre game play, even if the story and music are good to great.
I'll play a game with great game play, even if the story and music are a bit shit.

This feels very much like a 'click bait' title that's wanting to toss the bathwater out with the baby.
-Mind you, I hate children, but even I wouldn't waste water.

 

[Drathnoxis]

-Excessive Profanity
Swearing is permitted on the forums, but only in moderation within posts, and never within titles of topics.

Nice to see the staff setting a good example here. 10/0

 

[Bobular]

I had a simmer problem when I had my first performance review at work, we had to rate ourselves out of 10.

I gave myself something like a 8 or a 9, having being taught at the university I had just left that anything below 7 was in drastic need of improvement and since I had only just started there wasn't any kind of problems other then inexperience. My manager gave me something like a 3 or a 4 and had a go at me for thinking I was almost perfect when I had only just started and there was no way a trainee could get as high as I had put. When I told them that's how university told me to do these sort of things I was just told, "Your not at university anymore".

In my mind for me to get a 3 or a 4 at the start would basically have meant I had fucked up everything I had attempted, turned up late everyday and been rude to the other staff which defiantly wasn't the case, but to my manager it just meant that I hadn't learned enough about my job yet to warrant a higher score and it was a starting point to improve on throughout my training with them.

So since then I have always understood that review scores are basically meaningless without understanding the reasoning behind the score and if your going to go into that much depth, then I don't see the point of a quick score at the end making the score pointless.

 

[FirstNameLastName]

Scores are fine, people just need to stop taking them so goddamn serious, and see them for what they are; a simple numerical representation of the reviewer's overall enjoyment of the game (or movie, or whatever).

- 9 or 10 = great
- 7 or 8 = good
- 5 or 6 = average
- 3 or 4 = below average
- 1 or 2 = bad

It's no different from coming out of a movie and saying 'I loved it', 'I liked it', 'it was okay', or 'I hated it'.

Agreed. One of the main problems with review scores is that a lot of people see numbers and immediately think they can conclusively show that Game A is 21.8549353533% better than Game B. Even considering the different scales of reviewers, as long as you take the number to be a vague representation of how much the reviewer enjoyed the game, and take the reviewer's options as only a vague representation of how much you are likely to enjoy the game, then all is well.

 

[sXeth]

Scores are fine, people just need to stop taking them so goddamn serious, and see them for what they are; a simple numerical representation of the reviewer's overall enjoyment of the game (or movie, or whatever).

- 9 or 10 = great
- 7 or 8 = good
- 5 or 6 = average
- 3 or 4 = below average
- 1 or 2 = bad

It's no different from coming out of a movie and saying 'I loved it', 'I liked it', 'it was okay', or 'I hated it'.

The main problem is that the industry has evolved in an oddball way so that the review sites are supported by advertisements primarily of the things they're reviewing. While I'm not in the conspiracy camp of outright paid-for reviews, I'm prettysure if a site starts handing out too many things below that 7-10 scale, their advertising revenue is going to shrivel up like a prune under a heat lamp.

Hence 7 has become the bottom end of AAA, with lower scores usually reserved for indie shovelware that doesn't pay ads anyways, or the occasional uber-bombs that can't be passed off as anything else.

 

[Zenja]

If scores don't matter neither do the reviews. As the article mentioned any review that uses generic terms like "formulaic" and other buzzwords without in depth explanation as to why could still end up suffering the same fate as the arbitrary numbers. Someone could write an entire page about a game using generic buzzwords and it would mean as much as someone who just said "I give it a 7". There is nothing inherently wrong with number rating stuff. The biggest problem I have with it is that Halo 7 isn't even in production yet, but I can predict it will be a 9.5 or a 10 when it does come out even if it is shit. Look at Oblivion, Dragon Age 2, Diablo 3, etc. Some games get handed a score of 9 or 10 just for existing. As well as a page or two of glowing reviews and total dismissal and downplaying of problematic features and bugs. The more popular a franchise release, the more likely it will be handed a 9 or a 10. Also the more likely reviewers will use generic buzzwords and phrases to substantiate why the praise it.

 

[Pyrian]

I would like aggregators to include some measure of variance, like a standard deviation or something. A score of 70 +/- 30 is very different from 70 +/- 5.

 

[Silvanus]

There's nothing inherently wrong with applying a numerical score in a review; it's just shorthand, giving people the chance to get an impression of the general popularity of something without doing extensive reading.

What's wrong is the attitude towards review scores. They contain almost no information, and no nuance. They're meant to act as a brief summation of a longer argument, not a replacement, so if you look at the number and then back out of the article you're doing it wrong (and doing yourself a disservice at the same time).

Review score aggregators are inevitable, but the ones we have now tend to colossally mismanage their job by applying numerical values to reviews which didn't feature them, and by giving an average. A numerical average on an aggregation site for reviews is completely meaningless. Worse than meaningless; actively misleading.

 

[KingsGambit_v1legacy]

I like review scores. Taken singly of course they may not mean a great deal, but when combined with the review and/or any combination of word-of-mouth, aggregate scores, critical/fan consensus, experience with demo, experience with company, etc I think they're totally valid.

There cannot be an "industry standard" nor can a review be truly objective. Who seriously wants "truly objective" reviews? A reviewer must pass their subjective judgement on a game, that's what they do. Do the graphics sell the story and immerse one in the world? Do the sounds and voices do the same? Is the writing good? Are the characters believable? Is the game enjoyable? These are FEELINGS, not quantifiable, objective things to be measured. Reviews can and must be subjective by the reviewer and we, the reader can only decide whether to believe it and whether we would like it based on the review.

By objective, I always assumed it meant "with no stake in the game's success or failure". eg. The reviewer does not work for the game's developer or publisher, they don't work for a company paid by the dev/publisher to advertise it, nor are they married to or sleeping with anyone involved. Assuming a reviewer has no personal stake in a game's success or failure, their subjective review will be "objective", at least enough for me.

Anyway, games are subjectively good and bad with varying quality of graphics, sound and gameplay. I know for a fact that I could quantify my opinion on games numerically and I value review scores on the whole.

 

[CaitSeith]

Games are a combination of technology, art and product. Sometimes these characteristics enter in odds with each other. Finding the common ground from where you can evaluate all 3 at the same time is already difficult (let alone give a score). But that isn't really the problem. The issue is that different people gives different importance to each component. This doesn't include just reviewers; readers also discern between them about what matters the most in the same game (thus introducing subjectivity in reading a number).

So score aggregation will keep being messy, until everybody agree in what's important in a game and what isn't. But, would that be really a good thing?

 

[IamLEAM1983]

Scores are fine, people just need to stop taking them so goddamn serious, and see them for what they are; a simple numerical representation of the reviewer's overall enjoyment of the game (or movie, or whatever).

- 9 or 10 = great
- 7 or 8 = good
- 5 or 6 = average
- 3 or 4 = below average
- 1 or 2 = bad

It's no different from coming out of a movie and saying 'I loved it', 'I liked it', 'it was okay', or 'I hated it'.

That makes sense to someone who reads a lot of reviews, but what is the baseline of "average" for someone who does not play a lot of games, or has no reference of best and worst parts of a genre (racing games for example). It is not like review sites have guides on how to read their scores, and how they are different to other site's ratings. I mostly agree with how you placed the numbers, but from what I see, IGN considers 7 to be average, and Polygon places its average at 5, and that's not even mentioning the quality of the written review can be very different as well.

All of that's without mentioning how these numbers are ultimately understood in different ways, and value more or less for each gamer, all depending on their attachment to the game that's being criticized.

You've seen it before, after all: Anticipated Game X gets an average score, with an honest and thorough analysis in the text to give weight to that score, but Game X's defenders attack the reviewer for not conforming to their expectations, lobbing the usual salvos of "paid review" or "bullshit journalism".

Someone with more distance could take the same score and understand that the game in question deserved it, but we've all seen instances of gamers with zero critical distance and an excessive amount of brand loyalty.

 

[AzrealMaximillion]

Scores need to piss off. The defense of scores are also pathetic. "The review is subjective". Then don't put an objective number on it. I hate to say it but Kotaku has done something right. In ridding themselves of score they force a smarter look at their critique of games.

My main critique of scores of that the positives and negative of a game change wildly. Not just by who reviews the game, but even game to game by the same reviewer. Like someone who doesn't like GTA for having bad guys as protagonists but at the same time loves how badass the child soldier using Big Boss is in MGS.

I disagree with the idea that numbers are inherently objective. For that to be true, they must share the same context.

They do in the broad sense that they are presented in within gaming ads and metacritics. So much so that some titles "The MGS Collection comes to mind" advertise their metacritic score. You'll see the score from a website without anything but a quote that's not even a complete sentence plastered into launch trailers. When you read the actual reviews you find the number has no real cohesive meaning, but the way the industry advertises makes them see more objective.

Its why it irks me that Bethesda and Total War titles get perfect scores despite having rough launches that take months to patch.

 

[Fiz_The_Toaster]

The only time I will have a problem with a score is after reading the review and the number doesn't watch.

Like, the written portion drags the game through the mud and the number is a 7 out of 10.

Basically inconsistency, and that's about it. The number just gives me an idea of what the reviewer thought in some numerical value outside of detailing their justifications on why they had certain problems with the game.

I enjoy both portions of it. There are a few times where I won't have time to read a review and I want a 'down-and-dirty' answer. Later I will read it to see if that number makes sense to me. It also depends on who is doing the review. I mean, I'm not gonna put much salt behind a reviewer that doesn't like horror games and reviewed one. That's just me though.

But yeah, I like numbers. Numbers are awesome.

 

[Tanis]

-Excessive Profanity
Swearing is permitted on the forums, but only in moderation within posts, and never within titles of topics.

Nice to see the staff setting a good example here. 10/0

You, I almost like you.
If only your avatar wasn't Rainbow Dash being hung like a horse.

Also:
Why can't more reviewers be honest from the start?
Just start with 'while I'm not really into *insert series/genera here*, I'll do my best to review *insert game here*'.

I've read more than one review that's been done by someone who doesn't 'get' the genera.
AT.ALL.


'The new Korra/TransFormers/TMNT game is only 4-8 hours long, AND they expect you to beat it more than once!
OMG HOW LAMO IZ DAT YALL. Also, no multi? And where's the dubstep?!?!!? 4/10 bro!'

 

[NPC009]

Scores are fine, people just need to stop taking them so goddamn serious, and see them for what they are; a simple numerical representation of the reviewer's overall enjoyment of the game (or movie, or whatever).

- 9 or 10 = great
- 7 or 8 = good
- 5 or 6 = average
- 3 or 4 = below average
- 1 or 2 = bad

It's no different from coming out of a movie and saying 'I loved it', 'I liked it', 'it was okay', or 'I hated it'.

That makes sense to someone who reads a lot of reviews, but what is the baseline of "average" for someone who does not play a lot of games, or has no reference of best and worst parts of a genre (racing games for example). It is not like review sites have guides on how to read their scores, and how they are different to other site's ratings. I mostly agree with how you placed the numbers, but from what I see, IGN considers 7 to be average, and Polygon places its average at 5, and that's not even mentioning the quality of the written review can be very different as well.

Most of the sites and websites I write/have written for have/had a page what explained exactly what their scores are supposed to represent. These pages take a maybe minute to read.

On the other hand, one of the first things you learn as a game critic is that many people don't actually read your reviews and just skip to the conclusion or score (maybe just the score)...