A Review Scoring System That Would Work

[irishda]

I agree with the other people in this thread. Scores are fine. After all, we can't expect people to understand there are nuances and subtleties that might affect people's gameplay experiences in a way that's not easily categorized into a numerical ranking system. And that small details are just as important as large ones if they affect people in big ways.

So yes. Let's keep a scoring system. Because if I want to disagree with a criticism/review, I want to be able to do it by dismissing an arbitrary number, not actually engaging the criticisms.

 

[Zhukov]

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

This is pretty much why everyone reads reviews though.

Hell, we see people getting mad over reviews that fail to confirm their feelings on games they haven't even played yet.

The only time people read reviews for actual information or opinions is when the review is of a game they haven't heard of and they're reading it out of curiosity.

 

[DrOswald]

I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

The assumption that people only use review scores just as internet flame ammo and to confirm their preconceived notions is a very shaky assumption, often asserted but never backed up by any evidence. Frankly, you stating that assumption says more about what you think of people than how people actually are.

In other words, just because you are a cynical bastard doesn't mean everyone else is an idiot.

http://xkcd.com/610/

 

[shrekfan246]

I don't see the problem with review scores. They're just very rough approximations of how positive the reviewer feels toward whatever they're reviewing.

It's only ever an issue when people expect the system to be perfect. I can give a book a 4 out of 5 because I think it's really good but not the most amazing thing ever. My thought out opinion is always going to be more useful, but 4/5 is a nice way to put those thoughts in context.

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

Of course, that point of view comes with the issue of not having an easy fix re: "get rid of review scores" and/or "read the review", because we would be trying to change the way people themselves actually act. And people generally seem to be fairly stubborn and resistant to change.

The assumption that people only use review scores just as internet flame ammo and to confirm their preconceived notions is a very shaky assumption, often asserted but never backed up by any evidence. Frankly, you stating that assumption says more about what you think of people than how people actually are.

In other words, just because you are a cynical bastard doesn't mean everyone else is an idiot.

http://xkcd.com/610/

Yes, I have a very low opinion of the people who are often the loudest within the gaming community. I am very comfortable admitting to that.

However, I did not say that people only read reviews to strengthen positions they already held; I said that the people who only read reviews to throw tantrums (or, alternatively, to smugly scoff about how right they were about how good/bad something is) are the ones I view to be the big problem with reference to review scores.

EDIT: Edited for a word choice.

 

[shrekfan246]

I rather firmly believe that the problem is the readers. Yahtzee touched on it a little, but it's the people who come into a review just to reaffirm their preconceived notions; when they see a number they disagree with for whatever reason, they have to throw a massive tantrum all over the internet because apparently other people holding different opinions is just something they cannot reconcile with their world view.

This is pretty much why everyone reads reviews though.

Hell, we see people getting mad over reviews that fail to confirm their feelings on games they haven't even played yet.

The only time people read reviews for actual information or opinions is when the review is of a game they haven't heard of and they're reading it out of curiosity.

I can't speak for other people, but I read reviews to get information even if it's a game I've been following.

More often I watch videos of the game in question, admittedly, as there are few reviewers or critics I follow who regularly put out written content, but I even read Jim Sterling's review of Final Fantasy X HD before I bought it, despite it being a re-release of a game I've owned for over a decade.

Honestly, sometimes I really do feel like I'm some kind of robot when it comes to the area of review and critique in video games, because I just can't fathom how incensed people get over these subjects. It makes no sense to me.

 

[CaitSeith]

I always translated scores into grades from school.
6 or below is failing, thus not worth buying.
7 is barely passing, but decent. Wait for a bundle release with other 7s.
8 is good, but not worth full price. Wait for a discount.
9 is amazing and worth full price, but no extras and debatable if worthy of getting at launch.
10 is PERFECT. As in flawless. Pre-order it, get the DLCs, get it at launch, have its children, play the living hell out of it forever and ever.

As a side note, no game has ever been perfect, no matter what marketing departments pay for.

That's kinda of a problem because the reviewers are the ones who decide what their numbers represent. And as there is no official standard on that matter, lots of conflicts and misinterpretations appear. For example, this is the IGN's version (rounded down):

0 - disaster
1 - unbearable
2 - painful
3 - awful
4 - bad
5 - mediocre
6 - okay
7 - good
8 - great
9 - amazing
10 - masterpiece

Now, their most famous score is the joke of the Internet (Pokemon ORAS: 7.8 - too much water), even if their score means "good".

Someone once stated that the gaming community treat the 7/10 games the same way as the movie critics treat the 2/5 stars movies. The difference is that lots of 2/5 stars movies are really popular with the general public (or they just don't care about the ratings), and that isn't the case in gaming.

 

[Jburton9]

Over the years I have taken the approach, there is the review commentary and then there is the review score. I read the review because I want to know what the reviewer thought about the game and their experience with it.

When the Gamespot mess happened and a reviewer was fired over a review score he gave, I started seeing a disconnect between what was said and what score was given. Pretty soon most games received a 7 or higher or the magical 8.5 Generally I find that if the game came from a publisher that did not supply advertising $$$ then maybe the score is more reflective of the actual review.

So any kind of stars, marks, chits, thumbs up etc is fine provided that the real review is still written in the review itself. We can leave the rating wars to those that are too busy storming the castle over number - raaahhh 7.3!?! how dare you sir! Have at thee!

 

[CaitSeith]

I agree with the other people in this thread. Scores are fine. After all, we can't expect people to understand there are nuances and subtleties that might affect people's gameplay experiences in a way that's not easily categorized into a numerical ranking system. And that small details are just as important as large ones if they affect people in big ways.

So yes. Let's keep a scoring system. Because if I want to disagree with a criticism/review, I want to be able to do it by dismissing an arbitrary number, not actually engaging the criticisms.

I can't tell if that last sentence is a sad truth or pure cynicism...

 

[CrystalShadow]

I recall several old gaming magazines that had subscores for their reviews. Not detailed enough to cover what yahtzee is suggesting, but it did remind me of them...

They had an overall score and a breakdown into 4 or 5 areas for things like graphics, gameplay, sound and story.

While still nowhere near detailed enough to give you any truly meaningful information, it did mean you could, at a glance get an idea of say:

Game 1: Really shitty graphics, but fun gameplay.
Game 2: Very, very pretty, but has a terrible story, and is really quite dull to play (or outright broken)

Not that it really helps much, but it was interesting...

 

[Grimh]

I think a 10 point scale just has this predisposition to become skewed. A 5 just inherently looks like shit to a lot of people.
I've seen this in other places where such a grading scale has been used.
It was almost always scores between 7 and 9 and when it dipped lower to 5 or 6 people around me reacted to how low it was.
Anything below 5 was pretty much never used.

I think it might just be something we just naturally do. I'm probably wrong though.

 

[StriderShinryu]

I still feel all these years later that the best numerical review system (and the one that truly worked) was the old EGM review system. Yes it was a standard 10 point scale with half points in between, but it included written reviews by 4 different reviewers. Each reviewer provided their own score and their own reasons for liking/disliking the game as well as having the opportunity to counter comments made by the other reviewers. This sort of approach doesn't really fix the problems found in a numerical (or award) based review system, but it very clearly displays that games are experienced differently by different people with different tastes. In EGM, iIf a game got really high scores by every reviewer you knew you were likely looking at a very high quality title.

 

[Evonisia]

I feel like review scores get even more messy in complicated because of the response to them. Some people go by basic logic of how numbers work (i.e. 5 is acceptable as it is in the middle), whereas some others will go by the education logic of 7 being the bare minimum which is insane to somebody like me who sits in the former category.

I do like looking at a review score just fine, but I don't see how they add to the actual text so I very much agree. Seeing Jim Sterling give the 8/10 to ZombiU after reading his review just gave me a very brief "aww yis" feeling, nothing more.

 

[K12]

The only reason review scores can be bad is because people treat them as if they have to be an objective unchanging definitive way of ranking every single game in comparison to every game. If you are only using the score as a judge of whether you might like the game or not then you are an idiot.

Any review is likely to contain negative and positive aspects of a game and it's sometimes nice to have a score as part of the summary. Not an Essential aspect, nor is it Recommended or should it be Avoided. It's just an option.

 

[K12]

I feel like review scores get even more messy in complicated because of the response to them. Some people go by basic logic of how numbers work (i.e. 5 is acceptable as it is in the middle), whereas some others will go by the education logic of 7 being the bare minimum which is insane to somebody like me who sits in the former category.

I do like looking at a review score just fine, but I don't see how they add to the actual text so I very much agree. Seeing Jim Sterling give the 8/10 to ZombiU after reading his review just gave me a very brief "aww yis" feeling, nothing more.

Just as a little aside here but in British Universities a mark of 70% is a "1st" which is actually the top grade boundary (there's also "*1st" or "double 1st" but lots of universities don't give those). The standard pass mark is 40%.

 

[StreamerDarkly]

I find this article to be full of terrible logic that attempts to support a preconceived conclusion, which, ironically, is something the author complains about in attempting to explain why review scores are bad.

The worst flaw is probably the idea that aggregate statistics, such as an average score, "merely flatten things out and become meaningless". To address this problem, it is claimed that we really need to see every single data point from some 20 or 25 independent reviewers to derive any meaning. Clearly not the case, as data sets of any size can often be well characterized by just two parameters: mean and variance of the sample.

Actually, the initial claim that 'averages are useless' because things get 'flattened out' is the sort of cringeworthy insight I'd expect from a 2nd year general arts student. Averages are great in certain types of analysis, particularly in the case of video games where the little niggle that would cause one player to dock marks is completely irrelevant to 90% of the gaming population. More precisely stated, every observation (individual review score) in the data set is contaminated by noise (personal preference that doesn't transfer to the majority of other players), and the only way to extract what we might call the signal (more objective assessment of the game's quality) is to compute the average rating. Obviously not everyone will agree that the average score reflects the game's quality, but it is the single number that MOST players will agree with.

To think you can just say "art is subjective" and be done with it is the height of ignorance.

 

[RedDeadFred]

I kind of agree that scores are meaningless. Different people have different tastes and also have a different idea of what their assigned number means. For example, the guy writing at Quarter to Three has been known to give very low scores to games that were generally praised. However, he bases his scores off of his own personal enjoyment and not necessarily the functionality of the game, or whether it may appeal to others.

Still, people will often point to Metacritic as the be-all end-all of determining a game's exact quality. What I find works best is following several reviewers who you tend to agree with (Jim Sterling in my case), or at least reviewers who give extensive videos where they layout their thoughts honestly and show that they're willing to call games on their bullshit (Angry Joe and Total Biscuit).

Personally, I like Total Biscuit's way of reviewing the best. He will explain exactly what he like and dislikes about a game. He'll then describe the type of person who might enjoy the game or possibly be putt off by it. He doesn't give a numerical score, he just gives his informative analysis of the whole game.

 

[Thebazilly]

I think the best way of dealing with game reviews is to find a reviewer that you like or usually agree with, then listen to their opinion. If you find someone who has similar taste in games to you, then you can take them seriously when they tell you to buy or avoid a game.

Even if your tastes don't line up 100%, you can still say to yourself "Well, Yahtzee hated this game, but he hates JRPGs in general, so maybe I would still like it."

 

[Darkness665]

Well done, the finish was perfect. You managed to actually convey a really good solution. The problems with it as well as a simpler solution.

What I would love from a review site is a review (from a proper reviewer: Yahtzee, Shamus, Ben Kuchera, a few others) although backed by some actual detail.

Size of download/install size. Roughly, in GB or MB. Titanfall: you really couldn't solve multiple copies of audio files?
System used: Xbone, PSfur, Kids handheld, phone, touch, whatever.
Will it run from the DVD, hard drive, browser or the mysterious ether?
Stand alone, Multiplayer, story mode, MMO/RPG/FPS/TPS
Online provider: Steam, UbiSoft, Origin, MSFT, Fred's house of games.
Internet connection: none, registration only, game server, every stinking time you load a scene

Controls: KB/M, game controller, touch, well done, not well done
QTE: One, several, lots, OMG its fucking David Cage!
Cinematics: One, some, lots, Heavy Rain.

Then a review, written or video. I am finally fine with video reviews but only because Yahtzee has shown that they don't have to be a Let's Play and teenage rage fest.

That's all, so I cut 90% of the A4 pages and left room for a proper rant.

Finally, please let me search. I don't own any handheld gaming devices, except iSomthings, so I don't care about 3DS unless my reviewers think I should know about a game regardless of platform.

 

[MrHide-Patten]

The purpose of the review score is to tell us whether to bother to read the review for a game we weren't interested in.

9-10: Wow, guess I should read the review to figure out what's so great about this. Maybe I'm interested now.
1-3: Ooo, guess I should read the review to enjoy the train wreck.
4-8: I'm not reading the cunting review.

Review sites could achieve the same effect by going with better titles or ledes. '[Game]: The [Site] Review' doesn't really do it.

Usually a good review can paste out the tone or quality of a game from the first couple of lines. Ergo Jim Sterling's Review of Dying Light; 'Thumbs Down At Sundown, A promising game, brought low by its own lack of ambition.'
A neat to the point summery of banality.

There's too much meaning attached to numbers to the point that if a Nintendo game gets any lower then a 7 the whole internet erupts into wailing dipshits.

 

[coheedswicked]

This is why I like your video reviews, Yahtzee. I don't have to read anything and you pick everything apart quite neatly ( and harshly).