Jimquisition: Review Scores Are Not Evil

[oskalaboska]

I don't really understand why the scores need to be so exact like 1/10 or even on a 100 scale, when their is to much subjectivity in that number. The only thing that is matter that is good, ok or bad. The one example of that (the reason why I stopped reading Jim's reviews) was Witcher2 and Skyrim. U see Jim criticized Witcher for different reasons and voila in Skyrim in some of the same areas was doing MUCH MUCH worse but there didn't count. Both games have cons and pros, both games are good, which are better is more a question of tastes (I liked more Witcher than Skyrim), so how does justify Jim score of a 6 and a 10. An 8 and a 9 that would be subjective but 6 and a 10, sorry Jim but I expect from a reviewer to be as objective as possible.

You've hit on why I don't follow review scores. I've read reviews where aspects (like shoddy cameras and repetitive levels) lowers a game to a 7 but another game as has the same issues and its still a 9.5.

Also reviews on multiplats are totally skewed. Just because you reviewed the 360 version of a game doesn't mean the PS3 version deserves the same score. There is no way that Skyrim for PS3 deserves that 10 (or 5) that it got.

Personally I think we should get rid of the number and just have the rankings be: Buy it Now, Buy it Later, Rent it, Never Look in this Games Direction.

 

[Sheo_Dagana]

I have always been a fan of the simple -/5 score system. It's very simple and gives you an idea of what the review is going to read like, without being ridiculously overstated. -/10 scores tend to exaggerated by quite a bit. Maybe it's because I was raised on GamePro, I dunno.

In example:

1 - Horrible, shoddy game that should never have existed. Technical errors, game breaking glitches, and poor graphics are just the start of where this went wrong.

2 - A game that is so bad that it is unplayable or not enjoyable. Games in this category usually have poor graphics, bad voice acting, a terrible story, or overly ambitious mechanics, and may also suffer from bad bugs. Should generally be avoided.

3 - A game. A mediocre title that is neither good nor bad, the game hits the mark of 'playable' and can be fun to play, but does nothing special to earn any recommendations. Usually something worth renting.

4 - An excellent game that is of above average quality. Great graphics, solid gameplay, and a good story, with maybe only a few technical problems holding it back from true greatness. Usually worth owning, but should maybe be rented first by those interested.

5 - An amazing game of superb quality. Everything from the sound, to the voice work, to the story, to the gameplay is incredible. Even a few minor bugs will go unnoticed, or will be forgiven, for the sheer amount of enjoyment to be had. A must-have for any interested.

That kind of thing. Obviously one could be more specific but that's just a quick run down of how I feel the -/5 system works and would/will personally use it.

 

[Aurora Firestorm]

Right, I keep seeing this and now I must point it out.

What do you mean by "Should be average". What on earth does ANYONE mean by that.

If the average game gets a 7 then 7 = Average.

The problem is that the average game gets a 7.

So unless a site specifically states that they use a different system from other sites etc et etc, 7 = Average.

There is no "Should" with averages.

Most people who hate review scores seem to forget that. Yes, it fucks up the system, but we all have internalized it. If something gets a 6 we dont think "This is an above average game" we think, quite rightly, "This game is below average". Because it is below the average that has already been set.

You see where I am going with this?

I have nothing against review scores. I believe them helpful. Nor do I have anything against either metacritic, well the review scores from professional critics at least.

I guess I hold a hated view these days. Yes, I accept that the system is a bit broken. But that aint the systems fault.

Yes. Yes there is a "should."

What is the average of 0-10? 5. Start with the hard numbers, not the opinions.

The average game should, therefore, mathematically be *set* at 5. Just do it. Statistically, most of *anything* that has a large enough population, will be average. Reviewers should give a game that you consider to be completely average and unremarkable but still playable, a 5.

At this point, fairly good games will be then compared to this 5, and they will get around 7 or so...but right now, *average* games are at 7. So you have a massive, enormous range of "how much games can suck" (because 0-6 is apparently considered bad), and a tiny range of "how much games can be good," which are three numbers. In three numbers, can you really describe how much one game is better than another? No. Not really.

We messed this up because we started out by putting early reviewed games around the wrong number. Average games were given a seven. I like the theory that it's because Americans think on the grade-school report card system, but who knows why. So if your first unremarkable games are given a 7, then the rest of your games are going to be compared to that, and we're off to the races.

If you start out asking what your average game *should* get, mathematically, you will start at the right point.

Set your ratings at average 5.

 

[Neferius]

"Review-Scores are HARMLESS" Pictured: Kirby aka. THE DEVOURER ...did anyone else find that ironic ?

 

[Atmos Duality]

The problem I have is that review scores are implicitly quantitative, not qualitative.

ANY review must be qualitative to have any meaning. You can't quantify an opinion, no matter how hard you try.
(and before some smartass comments about polls, that's counting a number of opinions within a population. The number itself cannot form the logical basis of those opinions. It's like saying "I believe him, because he used the number 3 in his review.")

So, I read a score of 10/10. This tells me you liked the game. Great. Fantastic.
Anyone can do that, even if it isn't true.

This tells me nothing about the game. Some people THINK it tells them is that it's an excellent game.
And those people are fucking sheep who don't actually think for themselves. Reviews are opinions. Not facts.

If I'm reading reviews to research a game, what good is that 10/10 to me? The READER?
As a summary opinion, it's valid (since opinions are subjective), but still quite useless. It's filler.
How do I know you aren't just typing that to suck off the publisher? How do I know you even played the game?
I can jump onto Metacritic right this fucking second and give any game a 100/100, without having played a minute of it.

OK, so lets assume you actually wrote a review, and you're attaching a score to it as a sort of summary.
Knowing that I JUST FINISHED READING/WATCHING YOUR REVIEW AND GET TO THE SCORE, I should already know your feelings on the subject matter, no?

A little number at the end isn't going to change what you wrote, and if you didn't write anything but that number, I logically have no reason to trust you. So, logically, I don't get why you would include a score except for either purely arbitrary reasons, or pressures from within your field.

In the absence of stupid business practices (like publishers using Metacritic as a basis for payment plans) I do agree that review scores are harmless...but because I think they're useless more than anything. Ignorance and the inability to logically recognize that fact is more harmful IMO.

I won't demand any critic change their style unless they are objectively misrepresenting their subject matter (flat out lying to the audience); as for review scores, I ignore them. They're useless to me because the score is arbitrary, and the scaling is arbitrary.

 

[God of Path]

Jim, I think it's worth mentioning, I nearly pissed myself laughing at the end of the video. Great job!

 

[Teh Jammah]

I can both support the arguement you make here Jim, and see the opposite.

On the plus side, an x/X or X% rating gives you a general snap judgement of the overall feeling the reviewer had (or by their wording should've had) for the product. And I personally tend to read/watch/listen to the full review to put the score in context. For example, Angry Joe gave Asura's Wrath a 6/10 but with the BASoA and his review explained the hows and whys, which basicllay translated into (as i interpreted it) 'good game, not worth the full price' (in fact I'm pretty sure he said rent it/wait for it to price drop). I did the latter, enjoyed my £20 experience, even went on and bought the DLC.

On a related note, prolly wouldn't've shelled out for Sleeping Dogs had the 2BFP of it not made it look like so much fun. Could go on, but, yeah.

On the other side you have the AAA bought & paid for reviews where a 9/10 is the minimum (and is in fact now the score for an 'average game with a good ad campaign' &/or latest popular franchise sequel) and it seems to exist to be plastered in ads to sell more copies of Clone Shooter 2013 Edition. THIS is the kind of score we all want to be rid of... and since it's, for better or worse, the dominant for of scores, thats how we end up with this week's episode.


... Also I like the new murder gloves. Although Defoe needs some more input

 

[aegix drakan_v1legacy]

For what it's worth, Jim, you're one of the few people who uses the review scores PROPERLY.

As in, beside the score, you say exactly what the score means. 8 being a great game with a few flaws, 6 being for genre fans only, etc.

A lot of other sites seem to arbitrarily slap the numbers on and make it seem that a 7 is a bad score, when it's really not all that bad.

 

[Jimothy Sterling]

Yes. Yes there is a "should."

What is the average of 0-10? 5. Start with the hard numbers, not the opinions.

The average game should, therefore, mathematically be *set* at 5. Just do it. Statistically, most of *anything* that has a large enough population, will be average. Reviewers should give a game that you consider to be completely average and unremarkable but still playable, a 5.

At this point, fairly good games will be then compared to this 5, and they will get around 7 or so...but right now, *average* games are at 7. So you have a massive, enormous range of "how much games can suck" (because 0-6 is apparently considered bad), and a tiny range of "how much games can be good," which are three numbers. In three numbers, can you really describe how much one game is better than another? No. Not really.

We messed this up because we started out by putting early reviewed games around the wrong number. Average games were given a seven. I like the theory that it's because Americans think on the grade-school report card system, but who knows why. So if your first unremarkable games are given a 7, then the rest of your games are going to be compared to that, and we're off to the races.

If you start out asking what your average game *should* get, mathematically, you will start at the right point.

Set your ratings at average 5.

No that is a misconception, an error.
In a class of students, you can have the average be a B+ if your students are great. Or well any score that THEY AVERAGE, this doesn't mean the AVERAGE SCORE should be the mathematic average of the scale. SO if there is a clear evaluation criteria, and games accomplish to effectively fulfill most of the criteria, they should get the numeric score that applies, and not the average, even if in general most games succeed to a certain measure.

IE: if most games are GREAT, you shouldn't give them an AVERAGE score, because they are the average quality of games, obviously.

 

[CapitalistPig]

Jim my boy. It must be a slow season. I love your show but I find the irrelevance of this episode almost as irrelevant as the 10 billion review scores that are shouted at me from TV commercials.

But since were on the stupid subject. The only review scores I've ever respected come from X-play which I've watched since it was extended play and it was just Adam (huh? anyone? yeah!) This opinion has been formed over almost a decade of watching this show, understanding the nature of the game reviewers, and overall being entertained by the whole process. I trust them because at this point I know what they like. When some commercial flashes up with whatever the latest pop/rap crap stating "review junkie gives this a 5/5" or "game monster weekly 5 stars out of 5." Well you get the idea. That crap doesn't matter. You have to know and trust a reviewer to respect their ratings. Therefore most ratings don't matter. And vice versa from Jim's standpoint I don't know why he cares about the trolls. Because if they don't trust his review numbers then they don't really have much business reading his reviews.

On that note I've watched this show long enough to say Jim probably gives pretty fair reviews.

Keep on keepin on Jim. But please find some new topics, these past few weeks are topics that only the darkest corners of the internet are even privy to.

 

[CapitalistPig]

I see much more complaining on the internet about how scores average around 7-8, and any game that got a 5 (which *should* be average) is seen as total crap, than how scores suck altogether. They're just weighted funny, giving them much lower resolution. Has any game ever gotten a 1? Ever?

Indeed. They do exist.

 

[LazyGlader]

~deleted~