Review Score Inflation

[Crunchy English]

All this talk of Medal of Honor has me thinking about review scores. It's all well and good to say "I don't read reviews" or "Review Scores don't inform my purchases". That doesn't really help, however, when development teams, publishers and other industry types use metacritic and other aggregate scores as one metric to measure their success. People's jobs and occasionally entire development cycles depend on this stuff. Medal of Honor is considered a "disappointment" by many analysts because it ONLY got a 7.5

What kind of score does a game needs these days to be considered a "success"? What's considered a failure? How would fix it?

 

[SamuelT]

I guess MoH was such a dissapointment with an acceptable grade because other games have done the same, only better. And you don't have to cash out ?60,- to have the same experience MoH gives you.

A game that stands alone, a different genre or something, that recieves a 7,5 would be a good buy, but perhaps not worth it to go stand in the opening lines. I think the dissapointment on a grade depends on what experience it's trying to sell.

 

[Koroviev]

I think it has a lot to do with exceedingly high expectations in this case. For many games, that score would probably be considered a success. However, with regard to this particular incidence, hype has proved to be a double-edged sword.

 

[tomtom94]

Review score inflation isn't as bad as it once was (go to Gamespot and check out the scores N64 games have compared to today's games) however it is kind of strange how only a game that gets 90% is supposedly a success.

 

[scnj]

7.5 is a perfectly acceptable score, but when you consider that Modern Warfare 2 and Bad Company 2 got in the range of 9's and 10's, it's a disappointment for the company.

 

[For.I.Am.Mad]

When we see 7.5 we think 75. 75 in most schools is a C and you're not getting college with a C average.

That's why, we subconsciously equate everything to school grading. When we see 5/10 we see 50 then we see F then we see FAIL!!!!!!!!!!!!!.

 

[Motiv_]

Y'know, you really shouldn't trust the big review companies nowadays anyway, at least I don't. Look around Youtube or check out some of the smaller review "Companies". They're bound to be much more truthful and usually will give you a longer and better look at the game. Or just watch some gameplay videos and decide for yourself. My 2 cents, really.

 

[latenightapplepie]

When we see 7.5 we think 75. 75 in most schools is a C and you're not getting college with a C average.

That's why, we subconsciously equate everything to school grading. When we see 5/10 we see 50 then we see F then we see FAIL!!!!!!!!!!!!!.

Um. Isn't 50% a pass? According to pretty much everyone?

And 75 is like 2nd class honours? At least, that's what my university says.

On topic: The scoring system for games is completely screwed. A film review that gives 3 out of 5 stars does not say that the film is bad, merely that it is average. And yet 3 out of 5 is only 60%.

I'm pretty sure The Escapist has done an article on this very topic at some point. If I find it I'll edit it in.

Found it: http://www.escapistmagazine.com/articles/view/issues/issue_260/7750-1984-Out-of-10

 

[For.I.Am.Mad]

Y'know, you really shouldn't trust the big review companies nowadays anyway, at least I don't. Look around Youtube or check out some of the smaller review "Companies". They're bound to be much more truthful and usually will give you a longer and better look at the game. Or just watch some gameplay videos and decide for yourself. My 2 cents, really.

The smaller companies are usually comprised of fanboys. It's double edged sword, if you go to the big guys who knows what deals they got going. If you go to the smaller dudes you get fanboy scores and IMO they're not nearly as truthful as the big guys. Fanboys are unleashed nerds, and nerds are nothing if not petty jerks.

 

[For.I.Am.Mad]

When we see 7.5 we think 75. 75 in most schools is a C and you're not getting college with a C average.

That's why, we subconsciously equate everything to school grading. When we see 5/10 we see 50 then we see F then we see FAIL!!!!!!!!!!!!!.

Um. Isn't 50% a pass? According to pretty much everyone?

And 75 is like 2nd class honours? At least, that's what my university says.

On topic: The scoring system for games is completely screwed. A film review that gives 3 out of 5 stars does not say that the film is bad, merely that it is average. And yet 3 out of 5 is only 60%.

I'm pretty sure The Escapist has done an article on this very topic at some point. If I find it I'll edit it in.

Well, I don't know what schools you're going to, but where I live, Philadelphia area, 65 and below is a fail. Must be a shitty college you're going to, heh.

 

[Ironic Pirate]

When we see 7.5 we think 75. 75 in most schools is a C and you're not getting college with a C average.

That's why, we subconsciously equate everything to school grading. When we see 5/10 we see 50 then we see F then we see FAIL!!!!!!!!!!!!!.

Um. Isn't 50% a pass? According to pretty much everyone?

And 75 is like 2nd class honours? At least, that's what my university says.

On topic: The scoring system for games is completely screwed. A film review that gives 3 out of 5 stars does not say that the film is bad, merely that it is average. And yet 3 out of 5 is only 60%.

I'm pretty sure The Escapist has done an article on this very topic at some point. If I find it I'll edit it in.

50%? You only have to know half the stuff to pass? My school has 65 (maybe 70) as passing.

Anyway, I think /5 works much better than /10 or /100.

1/5 is utter shit. Unplayable.

2/5 is playable, but marred by technical issues or poor design.

3/5 is average. Most games go here.

4/5 is good, but not amazingly so.

5/5 isn't perfect (nothing is), but it's still incredible. It advances videogames as a medium and displays incredibly amounts of polish. Expect non-gamers to be talking about this one.

 

[Motiv_]

Y'know, you really shouldn't trust the big review companies nowadays anyway, at least I don't. Look around Youtube or check out some of the smaller review "Companies". They're bound to be much more truthful and usually will give you a longer and better look at the game. Or just watch some gameplay videos and decide for yourself. My 2 cents, really.

The smaller companies are usually comprised of fanboys. It's double edged sword, if you go to the big guys who knows what deals they got going. If you go to the smaller dudes you get fanboy scores and IMO they're not nearly as truthful as the big guys. Fanboys are unleashed nerds, and nerds are nothing if not petty jerks.

That's usually true, but there are a few gems out there. ElderGeek is one I can think of right off the bat.

Moreso, that increases my conviction that you shouldn't listen to any reviews at all, just watch some gameplay, rent it if you want to, and make up your own mind.

 

[thenamelessloser]

Score doesn't matter that much when it comes to reviews for me personally. For instance, I really liked Alpha Protocol which got some bad reviews, but the things the reviews didn't like about it didn't bother me or even in some cases I actually liked what they didn't like. I do like reading pro reviews just because they come out sooner than others and even if their opinion sucks you can still tell certain things about the game.

 

[Imperioratorex Caprae]

I only have 2 scores for a game:

Good and Crap.

If its good, then I play it. If its crap, it goes bye bye. Simple, easy and doesn't make a mess when you try to go into detail why one game pwns the other. If you play it and like it it is therefore a good game.

 

[spartan1077]

I'd say 5/10 or 50% means a rent while 60% means a wait for price to go down. 70% should mean go ahead and buy but don't expect anything perfect and above 80% should be pre-order that sh*t.

 

[zhoominator]

50%? You only have to know half the stuff to pass? My school has 65 (maybe 70) as passing.

Anyway, I think /5 works much better than /10 or /100.

1/5 is utter shit. Unplayable.

2/5 is playable, but marred by technical issues or poor design.

3/5 is average. Most games go here.

4/5 is good, but not amazingly so.

5/5 isn't perfect (nothing is), but it's still incredible. It advances videogames as a medium and displays incredibly amounts of polish. Expect non-gamers to be talking about this one.

First, keep in mind that (at least here) anything below an A, about 75%, means shove all when it comes to getting into a university. Second, if you need to get that much to pass, then your exams must be a joke. Getting 50% doesn't mean knowing 50% of the course, especially in something where you have to write essays, where on one occasion I got marked down heavily because of apparently poor referencing despite using the system from the book we were told we had to buy. That stuff happens all the time at our uni. You could know 100% of the course and still get less than 65% because you didn't convey this knowledge in the way they like (which btw will vary from marker to marker).

With that said...

I pretty much agree with your system except for the use of numbers, if that makes sense. I'd say:


Avoid! - is utter shit. Unplayable.

Only rent if you're really interested - is playable, but marred by technical issues or poor design.

Worth a rent is average. Most games go here.

Definitely worth a rent/consider purchase - is good, but not amazingly so.

Almost certainly worth a full purchase - isn't perfect (nothing is), but it's still incredible. It advances videogames as a medium and displays incredibly amounts of polish. Expect non-gamers to be talking about this one.


Please excuse the plagiarism, I'm to lazy today to come up with my own ideas.

I think it's silly to make a "perfect" score because if you find something that is better, you look dead silly.

 

[Netrigan]

The metacritic score is pretty close to useless beyond identifying certain trends.

A score in the 80s or low 90s and that's a pretty good sign that just about everyone enjoyed it and lots of people really enjoyed it. Do a bit of reading and you'll have a pretty decent idea if you'll like it or not.

In the 70s, there's a sticking point. A bit of reading and you'll be able to identify exactly what it is. In MoH's case, it's a campaign that's too short with some unclear objectives. Again, a bit of reading and you should be able to figure out if said flaws would get in the way of your enjoyment.

In the high 90s... pretty much all bets are off, because we've entered thinly disguised fan-reviews. Too many games with obvious flaws get near perfect ratings because the folks reviewing the games are too excited about them and aren't being objective in their reviews.

 

[Atmos Duality]

It's one of the very few aggregate scores I've seen in the last several years that actually seems to match the context of the game, and how it compares to its current competition.
You know, those little nagging critical details that matter most to the consumer.

"Why should I pay full retail price for a brand new game, when last year's entry is virtually identical, and cheaper?" Is a perfectly valid argument to make.

Seriously. I have seen and played so many games that were hyped well beyond reason and beyond what they deserved (I could name examples, but that will probably be perceived as trolling, so I won't bother).

For once, it's refreshing to see hype fail, and I hope it leads to a wake-up call for the gaming industry; that call being, "STOP MAKING THE SAME EXACT GAME OVER AND OVER AGAIN".

 

[Breaker deGodot]

It's because a vast majority of reviewers seem to be afraid of using the whole 10-point scale. TVTropes calls this the "Four-Point Scale", and it's extremely damaging to the world of gaming criticism.

EDIT: Actually, here's a 12-point scale I came up with. It's not perfect, but it's what I think of (may be tl;dr):


0: Completely unplayable - Think Big Rigs: Over the Road Racing. Unfinished. Painful.

1: Abysmal - Think E.T. 2600. Enough to qualify as a "game", but not at all worth playing. Good snark bait.

2: Terrible - Really, really bad, however some fanboys may be able to delude themselves into playing it through.

3: Bad - Maybe has a decent concept, but was rushed before it could be fully realized. Not much fun, unless for die-hards.

4: Subpar - A game that might be fun, but better (or cheaper) alternatives.

5: Average - Middle of the road. Most games fall under this. May like it, may not.

6: Decent - Maybe slightly rushed, but conceptually strong.

7: Good - Fun for most everybody, and definately worth a rent. Maybe a buy.

8: Great - Buying is strongly recommended, because this is a truly exceptional experience.

9: Fantastic - Buying is a must. A gaming milestone, or perhaps an unexpected gem.

10: A Masterpiece - THE game of it's generation. Little to no detracting features.

11: Perfect - Doesn't happen, in fact I'm not sure why I even included this.

 

[CAPTCHA]

When I rate a game I do so by rating its individual components: sound, graphics, story, concept, etc. That way I can build up a good avearge for its score. Each game has its own purpose so each game get a different set of criteria under which to be to be judged.

Can't say I've noticed "official" review scores increasing however. Though user reviews are utter crap. Just take a look on xbox live arcade. Even the worst games have 4 star reviews and many games that haven't even been released have 5 stars. The "mass" is either stupid or biase.