Changing the numerical review debate

[StriderShinryu]

Youtube channel Rev3Games has recently decided to go back to a review system that does not include a final review score. Instead of a score, their reviews are set to include a sort of concluding statement that speaks to the qualities (or lack thereof) of the game without assessing it a numerical value.

We've all heard and had the debates over whether or not review scores are a good thing. The Rev3 staffers Tara Long and Nick Robinson bring up the usual arguments like how granular review scores raise comparisons that don't really make any sense or how score based reviews just add to inappropriate time/value arguments.

Nick and Tara, however, also discuss how using review scores only adds to the vitriolic commentary in response to reviews. I'm not sure that's an argument I've ever personally heard before. It's a common refrain for reviewers to complain when their reviews instantly become arguments over whether the game deserves a 4 or 5 out of ten or how a game got a "perfect" 10 out of 10 when no game is perfect. While this sort of commentary is, in a word, stupid, I have to agree that it's also something that the reviewers are really inviting by themselves boiling their reviews down to a numerical score. Where is the discussion supposed to go when your review conclusion takes "Game X has numerous flaws in it's execution but the ideas and advancements contained in the game make it worth playing for anyone even remotely interested in the subject matter" and instead just sums it up as 7/10? Rev3's Tara even mentions that the comments on her Transistor review, which uses the new scoreless format, are the best comments she has seen for years.

I'm curious what others here have to say about this approach. Does removing a numerical score instead frame the discussion around the actual review instead of just the number? Are reviewers themselves just inviting ignorance by choosing to use a number?

-----
For those who are interested, here's the Rev3 video where they talk about their "new" review system:
https://www.youtube.com/watch?v=C1COYhH9w50

 

[Eve Charm]

All it really means is people can't just look at a page for a second and find a score, if you actually have to read more then the headline and a number of course your going to have an more informational and better comments ((see how that wasn't for tomodachi life ;p))

Problem is I don't know how you can get on Meta critic or game rankings without a review score, and the major sites and magazines have so many reviewers that most people don't even bother looking at the author of the review or have much to compare them to to get to know how they usually rate stuff or if it's "Something they like or not"

It's something that would work for like a one or few person show on youtube or something but it'll never catch on to major media that needs those links from meta and game rankings. Also some people just want scores not an whole review of the game, especially if your spoiler wary.

 

[CloudAtlas]

Yea I can well imagine that dropping numerical scores is helpful here. However, I'm pretty certain that when a reviewer brings up certain issues in a game - the "-isms" - that the discussions will turn out just as ugly.

While I can't say I harbour a strong love for numerical scores, I have to say though that if the abundance of vitriolic comments was a reason to drop these scores, I would find that rather sad. Reviewers feeling like that had to act a certain way because their audience are just too much of immature assholes, that would be a really sad notion. It would just show what a pathetic, immature bunch many gamers still are.

 

[MysticSlayer]

I like review scores as a good, quick indicator of what the reviewer thought, but I hardly think we should base on our decision to buy or not off of them. I generally use them to quickly find a few reviews that were positive and a few reviews that were negative, and after actually reading/watching them, I then decide if it is worth buying based on the content of the review. The only issue really comes up when people hinge their opinion of the review off of whether or not they agree with the final score, but they'd probably do the same for the review if the content didn't agree with them even without the score.

 

[Vegosiux]

The numerical score is the equivalent of TL;DR, really. Now, why people wouldn't want to R the review even if it's a bit L, and get all the relevant information, that's beyond me.

 

[CloudAtlas]

Yea I can well imagine that dropping numerical scores is helpful here. However, I'm pretty certain that when a reviewer brings up certain issues in a game - the "-isms" - that the discussions will turn out just as ugly. (...)

it could be worse if they focus on it too much as then it's playing that as a major issue while with a score you can go on about these issues and then show that it barely impacted your view of the game / it's quality.

If Gamespot's GTA V review is any indication, then no, it hardly could be worse. Although I guess I shouldn't say that, for we should never undererstimate to which lows gamers are able to sink.

The reviewer there just mentioned the sexism in GTA V with a few words, which was enough to make some assholes' mouths fuming, calling her all sorts of horrible things, demanding she be fired, and so on (well, that and that she's a trans woman, of course). Nevermind she gave GTA V a 9/10 regardless.

And that's all reviews ever do and probably ever will. Mention sexism/racism/etc in a paragraph or so and maybe, if they're bold, saying something along the lines that the reviewer didn't like it, or that if you don't like that stuff it could impact your enjoyment of the game negatively. But apparently that's already too much for some.

 

[Liquidprid3]

Review scores with decimals are the ones I have a problem with. There's a clear difference between a 7 and an 8 out of 10, unlike a 7.6 to a 7.7.

What also grinds my gears is when people think that a review score is like a school grade. No, they aren't. A 7 out of 10 isn't bad, it's more like slightly above average. If reviews worked that way, than anything below an 8 would be bad. Reviews are more like analysis's or reccomendations, not test scores.

 

[Casual Shinji]

Yea I can well imagine that dropping numerical scores is helpful here. However, I'm pretty certain that when a reviewer brings up certain issues in a game - the "-isms" - that the discussions will turn out just as ugly.

While I can't say I harbour a strong love for numerical scores, I have to say though that if the abundance of vitriolic comments was a reason to drop these scores, I would find that rather sad. Reviewers feeling like that had to act a certain way because their audience are just too much of immature assholes, that would be a really sad notion. It would just show what a pathetic, immature bunch many gamers still are.

But at least people will have to take the review as a whole, and not just look at the review score and get pissed on that notion alone. It means they'll have to process the entirety of the review and base their opinion on whether they agree with it on that.

It also means we won't get any of that 'You said good things in your review, but still gave it a 7 out of 10' nonsense.

At this point, review scores seem to be doing more harm than good.

 

[LetalisK]

Review scores with decimals are the ones I have a problem with. There's a clear difference between a 7 and an 8 out of 10, unlike a 7.6 to a 7.7.

What also grinds my gears is when people think that a review score is like a school grade. No, they aren't. A 7 out of 10 isn't bad, it's more like slightly above average. If reviews worked that way, than anything below an 8 would be bad. Reviews are more like analysis's or reccomendations, not test scores.

But they do work that way. Anything below an 8 is seen as bad and it becomes a self fulfilling prophecy. Which is also how it pretty much works in academics too(despite a C being called average, it's anything but), unless you're in one of the ridiculously difficult disciplines where As are nearly impossible or an institution that uses bell curve grading. As far as I'm aware, those are not the typical cases.

In fact, I would not be surprised at all if the review score inflation is a direct result of being accustomed to academic grading. I wonder if anyone has looked into a difference in overall scoring between numerals and a 5 star system. I mean, they are effectively the same, but I think we have different conceptualization of the two systems that would lead to different overall scores, but the same impressions.

Edit: screwed up my quoting

 

[Zac Jovanovic]

Personally I was always a fan of "What was butt and what was not butt" system, too bad it never caught on.


But on a more serious note, I don't seem to care about the grading system because I don't really give any credit to reviews.

I usually just watch some unedited gameplay on youtube, I have several channels bookmarked who post raw gameplay of all newly released games without talking over it.

Most of the time I glimpse average user scores too. If they are overall bad I read a few comments to see what players' biggest gripes are.

 

[krazykidd]

I don't think we need to change it, but put a standard for it. The numbers should mean things. I'm not sure, what changed , if anything , but i used to trust the numbers at the end of reviews. But maybe i was just more naive .

 

[LaoJim]

Obviously there are arguments for and against review scores. I personally find they are useful, mainly because I can look at Metacritic and find dissenting voices amongst reviewers quickly (those who disliked a popular game, or liked one that was otherwise panned) and this helps add some balance to my buying process. But as others say, ultimately what is written in a review is often more important than its final score. (Though again being able to see a high review score on the main page of a website for a game I haven't heard of is a sure-fire way to get me to read the review.)

In any case its all rather sad that these reviewer are feeling bullied into removing scores, however if they feel their site is better and friendlier through removing them then who am I to say they're wrong. That said I'm not sure that if all reviewers ditched scores, the rabid fanboys would soon adapt and start to critise the content of the reviews as well. For example, you can say a lot of negative things about a game but still conclude that its fun. With scores you can emphasise this by giving the game a 9/8 out of ten. If you don't have scores, the fanboys may assume you are giving it 7/6 or lower and may react stronger against this than the scored review.

 

[RJ 17]

Personally I apply to Jim's notion of review scores: have the numbers actually mean something. If you're going to be a reviewer, make a little chart or something that clearly defines "Something that does a, b, and c is only worthy of an 8. But a game that does d, e, and f will bump it up to a nine. And games with w, x, y, and z deserve a full-on 10."

Even better was the system that Yahtzee mentioned some time ago: shift the scale a bit. Rather than having 1-10, make it -5 through +5. That way it's actually more clear to readers of the review that, say, a 2 (which would equate to a 7) doesn't mean that you think the game is bad (as most people seem to think that a 6 or 7 on a regular scale means the reviewer absolutely hated the game). It's still on the positive side of the chart so that tells the reader that the reviewer did enjoy the game.

I think that numbered scores can serve a purpose, but it's up to the reviewer to establish what that purpose is.

I will say that keeping your comments general and not giving a score of some sort to a game would be a nice method of keeping the "OMG GTA V WAS TEH BEST GAME EVAR HOW CULD U ONLY GIVE IT THR33 OUT OF FIVE *s?!?!?!!!!?!!!111111oneone!!11!" crazies out of your comment section. =P

 

[veloper]

I like review scores and the aggregate scores that rely on them.
These let me dismiss all the garbage (average low scores) quickly, so I can focus on the more promising games out there.

There are decent alternatives also. Some reviews conclude with a game is for such and so. Example: if you liked X, you'll like this game too, but if you hate Y-feature/bug you'll hate it.

Still, a critic sticking to his own personal opinion and just concluding with a score doesn't need to figure out his audience and will end up being more informative too, as long as he/she is consistent and even-handed.

What I'd like to see more in game reviews is direct comparisons with popular and similar games. "Better than X, worse than Y" could replace the scores and maybe even work in an aggregate.

 

[chikusho]

I think review scores mainly are a problem on the industry side than for me as a consumer.
I've been gaming for so long that I basically know what a review score means. I don't really read reviews at all, I just look at the score and a gameplay vid and can figure out fairly quickly if it's something that I'd like.

 

[squid5580]

The numerical score is the equivalent of TL;DR, really. Now, why people wouldn't want to R the review even if it's a bit L, and get all the relevant information, that's beyond me.

One simple word: SPOILERS

I just want to know if the game lives up to the hype and if the disc will self destruct when I load it. A numerical score gives me that. Although I don't see a need for decimal point reviews.

 

[Drops a Sweet Katana]

I think some sort of summary at the beginning of a written review could work, say a paragraph highlighting what the reviewer liked and disliked, and an overall opinion of the game. This could also be paired with a non-numerical score like "'s pretty good", "'s a bit shit really" and "HOLY FUCKIN' SHEEET YUSYUSYUSYUS!" and so on and so forth.

 

[duwenbasden]

I support that, because numerical scores on games is as stupid as the our feud with the Grey-Manes. Would you assign a score to the store-bought apple you just made, and compare it to the orange you had yesterday?

Fallout New Vegas is 84 on MC; Tomb Raider 2013 is 86. One is something I still very much enjoy, and the other I rather cover myself in dung than touch again.

What I'd like to see more in game reviews is direct comparisons with popular and similar games. "Better than X, worse than Y" could replace the scores and maybe even work in an aggregate.

A comparison to other games is the last thing you want if you want a peaceful discussion afterward. A bit ironic since you have a Vault-Tec display pic...

 

[And Man]

I don't mind numerical reviews themselves, I just hate that an 8 is generally seen as decent or average, while anything below is generally seen as poor. It makes review scores much less useful, basically boiling them down to 10 = Awesome, 9 = Great/Good , 8 = Good Enough/Average, 7 = Mediocre, 6-4 = Bad, 3-1 = Broken/Unplayable. There's really no point in using a 10 point system when only scores of 10, 9, 8, and maybe 7 actually matter, and everything below is just "don't play this". If it were more spread out so that 5 was considered average, or if the -5 to +5 scale RJ 17 mentioned was used, review scores would be a lot more useful, because you could better differential between really great games and truly amazing games.

 

[veloper]

What I'd like to see more in game reviews is direct comparisons with popular and similar games. "Better than X, worse than Y" could replace the scores and maybe even work in an aggregate.

A comparison to other games is the last thing you want if you want a peaceful discussion afterward. A bit ironic since you have a Vault-Tec display pic...

I didn't even think of that. More comedy posted underneath game reviews. I'm all for it!

The main advantage of this scheme though as I see it, would be readers getting a clearer picture of the critic's preferences and tastes.