Review Scores are Shit

[Zydrate]

I think people use the word "Formulaic" to convey that they've seen something far too often too recently and players get bored of doing the same mechanics for five years straight.

I have thankfully stopped looking at numbered scores. It's why I enjoy ZP so much and why I like reading reviews. Even Steam is more of a "Thumbs up, thumbs down, now explain why" system which I like. I want to see what's broken, see what works, and then judge some of my purchases based on that.

 

[jklinders]

Quantifying an opinion into a numbered scale just strikes me as weird. On things where a harder scale than like or dislike can be used I can see them. Performance, bugs, glitches, I can see stuff like that on a numbered scale but when you drift away from the hard stuff into opinion and likes and dislikes it gets a little more ethereal.

I won't refuse to look at a review on a numbered system. I get the reality of the industry and that assigning these numbers gets clicks and getting clicks is getting money. I will take them with a grain of salt though. I am nauseated that Metacritic scores are being used to determine bonuses to devs from some publishers. An arbitrary system like that being used to determine the bonus when a harder figure like...oh I don't know sales maybe, would make more sense is just weird to me. I've also I think gotten over the whole outrage thing about a game getting a disproportionately low score from a reviewer over stupid things like not understanding the game mechanics (Angry Joe, I'm still talking to you here about Alpha Protocol). Still, I cannot help but see these systems as a way to generate outrage and therefore clicks.

I'm basically trying to say that it doesn't matter to me too much anymore as long as no one gets hurt in the process. I've found a set of reviewers that I trust in the opinion of and numbered scores or not is pretty irrelevant to my trust in them.

 

[Callate]

Some points I agree with, some not so much.

I do find it more than a little odd that review scores are often scaled like school grades, where everything under 60% is a failure and anything in the 60-70% range implies a decided lack of effort. As Ms. Finnegan says, the makers of a game aren't solving for x; a game that scores a 70 hasn't gotten 70% of the problems they addressed "right". For students, perhaps, this makes sense; at the least, for consumers, wouldn't it at the least be more helpful to use more of the range? Otherwise you end up with an enormous berth that's merely different shades of terrible. 55% is execrable, 35% crashes half the time, 5% bricks every system on the network?

On the other hand, English teachers aren't necessarily just grading your essay on its grammar and whether it meets a checklist of a number of supporting points and a properly formatted bibliography, nor should they. That doesn't mean their opinion is no better or worse than anyone else's. I get more than a little weary of critics falling back on some variant of arguments like "it's just an opinion", "it's all subjective", "if you don't agree, look elsewhere", etc, etc. A critical opinion, whether it's expressed in a certain number of stars or a thousand words, should have some value. And a critic shouldn't be telling me that I shouldn't care.

It's true that a critical piece is a snapshot of its time. There are movies, books, pieces of art and works of music that were scorned at the time of their release that later are recognized as hits or "cult classics". Shakespeare and Dickens were pop culture in their day; now they're considered great literature by most.

But I think I should care what a critic says about the latest grey military first-person shooter if they say it's just going through the motions. And I should probably view that with more respect than the fifty die-hard fans who have been playing every iteration of the series for a decade who scream that it's great and the critic doesn't know what they're talking about. Unless I'm one of those die-hard fans, the critic's point of view has more value... And if I am one of those fans, I'm not looking for criticism at all so much as affirmation. (And that's a whole 'nother kettle of fish.)

Not irregularly I get to read people who defend things to the point of "If you squint, and turn your head sideways, and allow your vision to blur, and look through the window on the south-west corner of this building, you'll understand the beauty of this object". (Or you could substitute "Real gamers play this on hard difficulty at 720p so they can get 60 frames per second with the $200 special-order controller that was sold out after nine days.") Good critics both bring their knowledge to bear on a subject and see through the kind of crap that deforms the view of people who are willing to contort wildly in order to maintain their personal narrative.

It is indeed weird to come out describing the hard work of a hundred people over the course of a year as "average". But at least for the time, I think a sufficiently large group of intelligent and well-informed people with an interest in a subject are able to come up with something that resembles an actual "right answer"- whether that's expressed as a consensus opinion or a numerical mean.

 

[thepyrethatburns]

Meh.

The individual scores are useless but, as an aggregate, it gives me a quick way to look on Metacritic or Rotten Tomatoes and see if something is great/shite. Usually, when I am interested, I'll read both two low and two high reviews to get a feel for the pros/cons of the product. However, I have neither the time nor the interest to do that on everything that I feel a marginal interest for.

The real issue is that professional reviews on any gaming site are rubbish. Even excluding my own experiences, there have been enough Jeff Gerstmans out there to know what happens when a professional game reviewer goes too hard against the sponsors. To use a movie example: The problem with the professional X-Men: Apocalypse reviews is the sheer scope of how much Disney owns (Example: They own ABC and all it's subsidiaries/affiliates.) and a lot of those places have reviewers who are acutely aware of how expendable they are. So, can I really trust a Disney-owned reviewer to do a good job reviewing the movie? Probably not.

Frankly, I think the big problem is that game reviewers themselves are obsolete. They don't really have that much more insight into a good game than the "reader reviews" section does and they're certainly less informative than even the low end of the Let's Play video spectrum. As such, I would suspect that the whole business of professional game reviewer doesn't need to be overhauled so much as it needs to be tossed out altogether.

Sidenote:

"Review scores today rarely reflect the dynamic nature of games - ones that add new content, and the quality of that content, or one's with a major flaw that is patched shortly after release. "

GOOD!!! I realize that the consumers have long since resigned themselves to the notion of just bending over and letting the industry have their way with them but review scores should remain as a "Here is how it is out of the box." We have enough issues with the companies shipping broken crap and not even bothering to ever patch it without removing one of the few motivators they have to make the game actually work out of the box. Also, if the game is incomplete without paying another $50 in DLC, that should count against it.

Once again, if we want to drop this function, maybe we should just drop professional game reviewing altogether and just go with a consumer-driven review model.

 

[veloper]

I like scores and I think they make a useful, additional quality indicator in large aggregates, but if a reviewer doesn't want to use (precise) scores or cannot make up her mind, she shouldn't assign one.

Any coherent review will still tell the reader if the author loved, liked or disliked a game and why and that's the essential part. Every reviewer assigns a value with or without numbers, just not necessarily with accuracy.
We don't really need to know just exactly how shit the game was scoring between a 6 and a 1 anyway. If it's not a recommendation, it's not a recommendation to buy.

Just a decent review by someone who really knows their genres is what it's really about. The rest is extra.

 

[weirdee]

The problem is that even interpretation of the same numbers by different people can vary greatly, so while anybody can value a number greatly or not at all...it does not actually do a reliable job of delivering the message that the author intended to convey. Assuming that you know what they meant isn't the same as actually knowing.

I actually find Steam's review system useful, which while home to large piles of shit, also contains the most pertinent information I need as the reviews sort themselves: why somebody does or does not enjoy playing that game, and what they think regardless of the recommendation. I am able to get a better read on the situation by cutting to the chase, rather than attempting to quantify the ephemeral.

 

[Yeager942]

I feel like this article perfectly shoots the idea of game critics in the foot, while also fully justifying standing its ground because the foot had a knife and was trespassing in the wrong shoe.

Here's the thing, if gaming scores are meaningless, because an opinion is too complex to display in a simple arbitrary 1-10 scale system, then that's fair enough. Even factoring in decimal points, that doesn't leave a lot of room for nuance between an 8.8 and a 9.0.
Having said that, if all game critics are giving their personal subjective opinion, then doesn't that kinda' make the whole idea of a critique...meaningless? I mean the opinion of the game is scene through a lens you, I, Susan Freeman from 4 miles away, Captain Ventris of the Ultramarines and Dracula can't replicate, ie the critic's themselves, assuming of course Uriel hasn't become another damned YouTube Lets-Player.
So then by what right does any critic at all dare to offer their opinion, if their opinion is applicable to themselves and nobody else?

Now to counter my own point, lets say we grant such Daring Do-ness to the critic by trusting them. Fair enough. I think Jim Sterling has a pretty good idea of what makes games good and bad. So I read his reviews and trust him. But then, if you trust them, then why shouldn't they give a numerical score? After all, the critique and score are scene through a lens we've chosen to trust.

So I'd say if we're willing to say 'Critic X's/Aggregate site's opinion of games is valid to me' I think we have to follow that up with 'And X's numerical score/aggregate score attempting to sum up their opinion, however weakly, is equally valid'. I don't think its possible to value an opinion or site while also devaluing a score.

TLR: either you trust the site/reviewer, or you don't. How they choose to publish their opinion is meaningless.

I remember reading A.O. Scott's description of what a good critic is. It's really about starting a conversation with the reader, and being critic is being someone with a deep, abiding awareness of the medium they are critiquing and thus able to approach and describe their experiences succinctly to an audience. Whether something is good or bad is hardly the point

 

[hermes]

Review scores are good as summaries. They are not good as (and shouldn't be used as) "objective" comparatives.

Saying "this is 10/10 = the reviewer really liked that game" works. Saying "if this is a 8 and this one is a 9, it means he liked that one better" doesn't work most times. Even if they are from the same person, it doesn't work (they could be from different genres). Even if they are from the same genre, it doesn't work (they could be from different time periods). Even if they are from the same GAME, it doesn't work (they could be from different people)...

 

[Lightknight]

Review Scores are Shit

With no industry-wide standard for review scores, are scores themselves - and to a larger degree, aggregate sites - beneficial? The short answer is no.

Read Full Article

Liz, the score isn't as important so much as the reviewer's consistency. You will have natural biases and preferences when it comes to game genres, plot mechanics and even tolerance/appreciation of technical shortcomings/successes.

Over time, I get to establish how your preferences line up or disagree with my own but only if you are consistent in your reviews. If you can have drastically different opinions of similar games then I can't use your reviews in any meaningful ways. But if you are consistent, then even if we strongly disagree I can usually get a gauge for whether or not I'll like a game based on your recommendation.

 

[Cheesy Goodness]

<youtube=PjYieeQofR4>

Interesting that I Hate Everything posted something with the same sentiment just this morning. IHE's video articulates what I think about the subject a lot better than I can put into words, but I will attempt to say a few of my own thoughts anyway. If you watch the video, you'll see that my opinions pretty much match up with his.

Mainly, I've started to get annoyed at aggregate review sites as of late, especially the likes of Metacritic or Rotten Tomatoes. I've been burned by Rotten Tomatoes' glowing recommendations before, because huge scores would build up false expectations. I try to base what I see from particular reviewers now that share similar tastes.

Also, something feels inherently wrong when you boil down the worth of a movie or video game to a number. While I understand the point of a score, I think they are often counterproductive. People make such a big deal if something they like or dislike gets too high or too low of a score. The substance behind a review's words should stand on its own merits and not need a numerical value.

 

[Synigma]

I like the way Cinemarter does it here on the Escapist.
Read the Bottom Line and Recommendation, then use the 1 to 5 star rating with that context to determine how interesting the movie is. That provides a decent TL;DR review and if the movie sounds worth watching but you're not sure if you want to see it, read the whole review.

 

[DrunkOnEstus]

I'm going to paraphrase SuperBunnyHop here, as his offhand comment perfectly encapsulates how I feel about review scores and "metacritic culture": "I wish I could give REmake a 10/10 so that you would know that it's my subjective opinion that the game is objectively perfect". Aggregate sites are a fools errand of attempting to objectify a games value, and by using an array of reviewers that value includes cultural impact, game feel, glitches/bugs/server problems, and sometimes beautiful insight into how the reviewer was impacted by the game, crammed and distilled using a secret and arcane method to produce a perfectly objective number value, with all of the actually useful information hidden beneath.

A number that has denied bonuses to studios, caused studios to shut down, driven away potential buyers or even attracted buyers who would otherwise be uninterested. I get it, I really do, I understand the desire to simultaneously set aside and yet include humanity, bias, hype, and objective facts in order to have an easy number to assign to a title. I think the human mind likes that, it makes compartmentalizing the torrent of game releases easier on us and helps us place the game in a tenuous yet useful situation for us. According to Metacritic, nearly everything Valve has made, plus GTA V and Bioshock are the greatest PC games ever made. It's a common consensus that most people accept, but one could find plenty of people who can't stand playing each of those games. Is the number useful as a buyer's guide? As a leaderboard of "professional" opinion? Sometimes both, sometimes neither. It probably explains a lot of the joke "7/42 would play again" Steam reviews, sometimes not any more or less useful than a "legitimate" metric from a "professional" reviewer.

Thank you for an excellent and thought provoking article, from someone who aspires to be a reviewer and someone who also can't bring myself to attach a cold objective number to the end of my thoughts and feelings of a work.

 

[DayDark]

great article. I wouldn't mind if scores went away to instead show just a short abstract of the reviewers opinion and enjoyment of the game.

 

[Secondhand Revenant]

I don't care to look deeply into every last game and read the entire review every time. A score helps tell me if I may want to look into it any further.

 

[SummerHaze]

Grades are important. In fact, combined with a challenging course load, grades are a pretty important factor for college admissions. There are close to 1000 ones in the United States that admit between 50-75% of their applicants, meaning they are ranked as ?somewhat selective?. The long and the short of it is that grades are important. If you are early in your high school career, great, get studying. If you are a senior ready to begin looking for colleges to apply to and your grades haven?t been stellar, get studying. But also think about other ways to strengthen your application. But good writing skills is needed to write such essay that will catch attention. Unfortunately, I am bad at writing, so I asked for a little help in https://www.essaymasters.co.uk/ . Many schools look not only at grades, but also consider applicant as a whole in making admissions decisions.