Jimquisition: Review Scores Are Not Evil

[ConanThe3rd]

You call that a "Wrong!", Jim?!

 

[Sir Thomas Sean Connery]

At the end, I could feel Jim's raging dictator boner through the screen.....

I tend to agree. Scores reflect an opinion and can be quite fun.

What needs to be stopped is Metacritic and other sites that publishers put so much weight on. Yeah, it would be best if they just didn't, but as long as it exists, they will.

 

[geizr]

There's nothing wrong with a review score. There is something wrong with how review scores are often abused, and it's not just readers misunderstanding the context of the review score. It's also reviewers that have, over time, whether deliberated or accidentally, distorted the meaning of the score such that only a small span of the entire range are given meaning. The Hate out of Ten problem comes partially at the fault of reviewers who have, over the years, created the distortion that scores in the range of 7.0-8.0 are only average or mediocre, as opposed to the 4.5-5.5 range having that meaning. Back in the day, if a game was truly shitty, it got a 3/10, at best. Today, that same level of shittiness garners a 6.5-7.0, a range of scores that was once considered slightly above average in quality. Even further, a game having such a score would still often appeal to fans of the game's genre; I'm not sure if the same holds true today.

That segues into another basic problem that I perceive regarding the review score; it lacks context of personal preference. Just about every review score seems to be given an absolute context when the reality is that a game that is considered shitty or mediocre by one individual may be considered exceptional or superior by another individual. The difference is personal taste and preference. Review scores often don't take into account the skews in perception of quality that depend on the preferences of the target audience. Fans of a particular genre or series are more likely to have a more elevated opinion of a game in their preferred genre or series, whereas those who dislike a particular genre or series will have a much low opinion of the game. Those who are ambivalent are likely to be somewhere in the middle. No singular review score is going to properly capture this nuance.

It is for this reason that I think review score should really be done in triplet, one score for fans, one score for haters, and one score for the general ambivalent populous. Of course, this means you have to have 3 different reviewers for any given game and they must have the particular dispositions toward the game as required by the particular type of score they are trying to yield. The primary problem I see with this solution is that it is expensive (for the reviewing publication) and cumbersome (it's hard finding that mix of fan, hater, and "don't care" attitudes), meaning the review process slows significantly and can cost publications looking for rapid turn-around. However, in my opinion, this would allow a lot more context to be applied to the game and give a more complete view of the game's quality.

 

[veloper]

First, the length of a game shouldn't affect the review score by itself. Sure, $60 for a 5-hour game probably isn't worth it, but that has nothing to do with the quality of the game. If something like Portal, for instance, were released as a stand-alone $60 game, should it be considered a lesser game than if it was sold for $10? Letting price factor into a review of entertainment also damages the score by dating it; in a year, a game can drop to half its price. Should it be considered a better game if it does? If people really want to address price, they should do it, but not as part of a score. Set a price limit, or a recommended format, but don't consider it in your score unless the length, or lack thereof, affects the quality of the game itself, not its value.

This is a very good point. Prices drop all the time, especially on PC and length is only value for money (if the game is any good at all).
Game length should be mentioned somewhere, but it doesn't make an experience more fun. Some of us don't even want to waste much time.

And second, the review scales should not use other games as their base. This might sound stupid, so let me give an example. New Super Mario Bros released more than 3 years ago, and got scores in the high 80's. New Super Mario Bros 2 released this year, with scores in the high 70's. Most (but not all) reviews I've seen credit the lower score to the game being too much like its predecessor. What does that have to do with the game's quality? Nothing. It has everything to do with the game's value. I see the same thing when games are compared to one another. A game's quality shouldn't be dependent on what its competitors are doing, but on its own merits. This also dates the review further, because its dependent on games that were out at the time of the review. Its value should be what's affected, not its quality.

Now I'm not saying that other games shouldn't affect scores at all; in fact, by defining a critic's quality spectrum, they already are. But the level of engagement should be what's scored, not how much the game is worth. If they want to include that, they should score the game's value separately.

Trickier. A reviewer may want to reward originality somehow, but at the same time to a newcomer who isn't jaded yet, copy-pasta sequel X may still be worth their time.
Worse, a fresh experience may genuinely make the critic like the experience more. If this rule is to be taken to it's logical destination, then a critic would have to go something like: 'wow I loved this game! Hmm, maybe it's because it's something not done before, so I'd better substract a full point just to be fair to all the copycats that will soon follow'.

 

[Imp_Emissary]

Well done Jim. xD The end of the show killed me! HA!

I don't see how people blame review scores or Metacritic for the choices of the people who run a company they aren't a part of. Even if review scores and metacritic left, or weren't ever there to begain with the people at the top of these companies would just make up some other bullcrap reason to fire people.

Also, those gloves do look better.

 

[Mortamus]

I, for one, welcome our new Sterling overlord.

In reference to the removal of scores, it's as old an argument as any. It just has a different title. Generally, the idea of "Take away their [tool] and then they can't do [deed that is deemed awful by it's author]" has little to no sense. If someone wants to do it, they'll find another means to do so. Plain as that.

 

[theultimateend]

Review Scores Are Not Evil

When Jim Sterling isn't busy being the voice of a generation, he's a videogame reviewer -- one that's constantly told to abolish review scores. This is a silly request, hinging on the belief that review scores are an evil we simply endure.

Watch Video

So I work at a game publisher, the videos on the escapist are technically work related [plenty of folks here have a routine of game industry shows we watch, others use their free time in the day to smoke, different strokes].

Now with that in mind, so glad the only penis in this weeks video was a guy in a cock suit.

The CEO of our company walked by and got super interested in the game scenes and was curious what your show was. I like that watching this show at work is a bit of a mine field .

 

[Imp_Emissary]

Wait, what? This is an issue? People who complain about review scores, get 3/10.
I'm giving them a point for being able to string a coherent sentence together, or at least enough of one that we can understand what they're complaining about. They also receive a further two points for having the gall to express themselves on the internet.

Yes, because it takes a lot of guts to say something terrible on the internet. That's why it almost never happens.
>_>
<_<

On a more serious note: The reason people hate review scores/metacritic isn't really because of the scores themselves, but because some companies have said they will fire anyone who works on a game that doesn't get a high enough score.

But they should get mad at the companies who say that, not the scores, or Metacritic.

 

[PunkRex]

I miss the red gloves Jim, you looked like a facist magician.

Also, I thought you had already done this topic but I guess not... weird... anyways, what did Koonami actually say? It sounds freaking hilarious.

 

[SonOfMethuselah]

I don't mind review scores. I read reviews in a similar manner to Jim: score first, and then the full text, to see how the result was arrived at. If it's a game I don't really care about, and I don't, for whatever reason, have the time to read a full review, I'll look at the scores on a couple of different sites to see if they match up.

I also don't have a problem with using numbers. Most sites have a pretty well laid-out system for scoring games, so even when the numbers don't match up, the thoughts on quality usually do.

The only reason why I'd want to see review scores abolished is because then the HUGE FANBOYS who only pay attention to scores would no longer be able to look at a number, and then ***** and complain for months on end that the reviewer in question was paid to give it the score it got. Christ, that pisses me off to no end.

 

[xPixelatedx]

take their ball and go home as Konami did

...Wait, maybe I am out-of-the-loop here, but were you blacklisted by Konami because you said something mean about their game(s)!? Because if that's true I am going to be a little shocked and disappointed with them.

 

[Mythmaker]

This is a very good point. Prices drop all the time, especially on PC and length is only value for money (if the game is any good at all).
Game length should be mentioned somewhere, but it doesn't make an experience more fun. Some of us don't even want to waste much time.

Precisely. As a consumer product its length is relevant because you want the most for your money, so it should be mentioned. But as entertainment its length should have nothing to do with its perceived quality by itself.

Trickier. A reviewer may want to reward originality somehow, but at the same time to a newcomer who isn't jaded yet, copy-pasta sequel X may still be worth their time.
Worse, a fresh experience may genuinely make the critic like the experience more. If this rule is to be taken to it's logical destination, then a critic would have to go something like: 'wow I loved this game! Hmm, maybe it's because it's something not done before, so I'd better substract a full point just to be fair to all the copycats that will soon follow'.

A reviewer should always score the game on their level of engagement; if the newness of the game is something that makes the game more enjoyable, that should definitely improve the score. But a lack of newness should not detract from the score. If NSMB2, for instance, was as good as the first game in every way it should not score lower simply because it is unoriginal. However, because you can get roughly the same experience with an older game, its value should be lower.

 

[Aureliano]

I think the problem is that most of the 'X out of 10' outlets are secretly working on an X out of 15 system, where 15 would actually be a perfect game and 9 or 10 is just a bit better than average. And people get so confused by this that now they get pissed off at people like Jim for using the number system for its purportedly intended purpose: as a quick way to tell people how good he thought the game is in comparison to other games in the genre (i.e. games he has scored either higher or lower or the same number to).

Am I the only one who got a sympathy chub with Jim's narcissism boner at his own gloves? Fantastic Dr. Strangelove-type times.

 

[PunkRex]

i don't really have a problem with scores it's just that they make no sense. first of all i don't believe a complex opinion can be quantified numerically and if it could you would have to decide on a universal scale to use it on, which also makes no sense because different people value things differently.

and there is this weird thing right now where 7/10 is average

5/10 is funtional mechanically, which is not average, despite being the number between zero and ten, because consumers hold the industry to a certain standard. This is mostly thanks to the internet giving consumers the tools to have the developers by their dangly gubbins, and a game that does nothing more than function mechanically (like that X-Men game) won't turn a profit (it didn't). So 7/10 is 'average' in that it meets our standards, as in, it's entertainment that's entertaining, meaning it does more than function, which is what we expect, and have every right to at sixty dollars a pop.

5/10 is a car that runs, 7/10 is a car that runs well, and has air conditioning and a radio. It may not park itself, it may not be a hybrid, it may not have heated seats and a damn GPS, but it's what people consider 'average' in spite of cars not needing A/C or a radio to function.

It's not that complicated.

I can definatly see your point but these extras have become the standard due to the tech and goals within the industry. Im not saying that every game MUST include them, its very much up to what the creative people behind the game want to acheive, but as standards increase the bar gets pushed up. Isn't this how its always been? Im aware im being quite general here but this is artistic oppinion were talking about, even if I have no problem with the use of numbers everyone has there own way of doing so as this isn't technically maths as there is no universal formula behind it.

E.g. If I went out to buy a car and it didn't have a radio id mark it down, granted only slightly. I'd say at least 80% of cars these days have a radio, its become the standard, the absense of one would be a negative not a plus if it did have one.

 

[CaptainOctopus]

Scores are great, how else will the robots know what is good or not when they kill all humans and take over the world?

 

[Imp_Emissary]

I, for one, welcome our new Sterling overlord.

In reference to the removal of scores, it's as old an argument as any. It just has a different title. Generally, the idea of "Take away their [tool] and then they can't do [deed that is deemed awful by it's author]" has little to no sense. If someone wants to do it, they'll find another means to do so. Plain as that.

Sterling overlord you say? So does that mean instead of ruling over us with a iron fist he'll have a SILVER one?

Also, I agree with what you said about the issue.

 

[IamLEAM1983]

I never understood how people could cry foul at an eighty percent or a solid B. As much as I hate throwing the "Fanboy" label around, there's some people who can't understand that everything - absolutely everything - should stand up to criticism. If you can take a game you honestly adore and still find niggles and small issues with it, you're doing your job adequately. If you're so blinded as to declare this or that game the epitome of perfection or are unable to understand that review scales change from site to site and reviewer to reviewer, then you don't understand the point of criticism in any shape or form.

No game will ever be perfect, and a eight or nine out of ten still denotes a very respectable and even commendable success. I'll buy games that sink as low as six or seven, sometimes, and I understand that all reviews are ultimately subjective.

Consider this: a friend of mine absolutely fucking loves Dead Island. He compares it to a Mêlée-centric version of Borderlands, and he loves the way combat can be fairly difficult or unforgiving. I played the same game for three hours and couldn't find a speck of enjoyment in it. It felt badly put together, poorly structured and thought out.

Who's right, then? We both are. There's no point in him trying to convert me, just as there's no point in me calling him a poopyhead for daring to like what I don't like. You can disagree with critics, and critics *will* disagree with you.

Yes, there's always going to be thorny issues like paid-for reviews and the general impression that payola is a common practice in the industry - allegedly, at any rate - but I'd still rather think that professional reviewers are hired for their ability to be honest about their preferences. This is why most pro outlets aren't just the voice of one guy testing out various games - a JRPG buff might have zero patience for a Western shooter or a sports sim. Making sure you nab reviewers from all corners of the gaming rainbow keeps things fair and balanced.

 

[mjc0961]

Okay, I'll give you that it's not the fault of the scores that people so blatantly misuse them, and yes I would love some kind of actual stand against these people beyond "well this guy is a dumbass, time to put them on my block list so I don't have to read more of their stupidity". Personally though, I still find review scores to be useless. Saying that a game is a 7/10 tells me one thing and one thing only: that whoever gave it a 7/10 liked the game despite what they felt were only minor faults. It does absolutely fuck-all to tell me if I'd actually enjoy it. I actually have to read the review itself to find that out, and by the time I've finished the review, I don't need a score anymore. It's just an extra summary put in after the written summary that uses a number instead of words.

I still want to repeatedly kick the genitals of every person who insists that because a game got good scores I'm not allowed to dislike it and it's better than a game I did like if it got higher scores. Just today some complete and utter dipshit on YouTube, in response to me saying I didn't like Sly Cooper while also recommending Ratchet and Clank to a friend, insisted that I must have never played Sly Cooper because it got better review scores than Ratchet and Clank and thus is a better game than Ratchet and Clank. FUCK. OFF. That is not what review scores mean you useless cretin!

Jim should do audio clips rather than videos. I don't need to watch streams of clips from Japanese games I don't care about while at the same time listening to completely unrelated streams of Jim's occasionally funny/entertaining monologues.

...Then don't. This may come as a total surprise to you, but you don't have to watch the video just because it's there. You can open a new tab and just listen to the words while you look at something else. You can minimize the browser, turn off your monitor, or get up and do other things in the room while the audio plays.

Meanwhile, those of us who like the video, especially when it's a situation where the game clips help drive the point home (such as "Monster Boobs And Plastic Children", where I never would have known that the volleyball game he was talking about was that creepy if he hadn't been showing clips throughout the episode), can still watch them instead of having them taken away because the almighty Blood Brain Barrier dislikes them.

 

[mjc0961]

It depends on how you look at the 10 point grading scale. If you look at it like an academic grade, then 7/10 *should* be the average/mediocre game. That's how most reviewers see it, even. Just look here on the Escapist where 2 and a half stars is usually accompanied by a "don't bother" recommendation and anything below that is pretty much considered garbage.

Which is wrong, because if 7/10 is the average/mediocre game, then you have only 3 higher numbers to give a game a better score and 6 smaller numbers to give the game a worse score. This "7/10 means mediocre" nonsense is why people cry foul at 8/10's. 8/10 does not mean slightly better than mediocre, it means very good or great. 5/10 is mediocre, not 7/10.

E.g. If I went out to buy a car and it didn't have a radio id mark it down, granted only slightly. I'd say at least 80% of cars these days have a radio, its become the standard, the absense of one would be a negative not a plus if it did have one.

Exactly. For claiming that it's "not that complicated", he sure got it dead wrong and you got it absolutely right. Having a standard feature does not mean you did better than average. To go back to the car comparison, like you said, a radio is expected in every car, and having that is not worthy of a higher score. Not having one is worthy of a lower score, though. You'd have to go above and beyond with the radio to get a higher score, it would have to be more than a standard FM radio with a tape deck and/or CD player. Maybe it has a 5 disc CD changer, or maybe it has built-in satellite radio so you can listen to radio stations that aren't 75% idiot DJs talking about shit you don't care about and commercials. Then you get a mark up. Having a basic, average radio gets you a basic, average score. And not having one at all marks you down.

These "7/10 is the average because it should be the average" people really need to try out common sense sometime.