Is game reviewing 'broken' as a system?

[Denariax]

This is kinda funny. But here's the bottom line; Popular games are the ones that have big name developers. In the end, 'reviewers' will stamp it with golden money piles to think they get something in return, for example viewers, which will raise their revenue. The problem with this is that all of the world's population is comprised of self-righteous gits; they will never 'award' anything they don't like, or; dare I say it; will only award popular games for previously stated income. This includes myself, except I don't have a website.

I'm an autistic, schizophrenic, bipolar, sociopathic mess of a human being; so why am I the one always trying to think for others?

 

[surg3n]

Yes, definitely. I am far more likely to judge a game based on youtube videos, escapist videos, yogscast, and forum posts - which I guess is more like a community review. I just don't trust reviewers to give an unbiased opinion. At the end of the day, a review is just a well written opinion (arguably!) - these days we can summarise several opinions and get opinions from people whose opinion we might actually care about. Never mind which reviewer got paid to say this or that, I am more interested in how much Yahtzee hated on it, or I'll watch the yognaughts do a play through, and get a real understanding of the game.

Times have changed, it's not like we have to go and spend our pocket money on a games magazine to get an idea about new games - the internet has levelled the playing field, and reviewers will have to adapt... the greasing of palms will become pointless once more people realise just how corrupt reviewers are forced to be a lot of the time.

 

[GigaHz]

I've also noticed the change in scoring trends and I agree with OP. There is no point spread anymore.

Literally, any half decent game will get some variation of a 9, or a high 8 at the very least while anything excellent will get a 9.5 or above.

What is the point of rating just about every other game a 9?

If there are in fact a surplus of 9s, then scoring standards need to change to reflect that. Games that are now scoring in the 8-8.9 range should be 5s, games rated 9.0-9.5 changed to 6's to 8's, and lastly games rated 9.5-10s changed to 9s and 10s.

Obviously anything below that would be in the 1-4 range.

But who am I kidding? Something like this will never happen. Kids these days go into nerd rage when their favourite game doesn't achieve a 9. Would you imagine if they had to *gulp* consider purchasing something that might be a 6 or 7? Oh the shame!

 

[predatorpulse7]

I think there is something broken/corrupt in it when almost every AAA game rating nowadays is from 8.5 upwards. No doubt, some of them have high enough production values to ensure a high quality game but a lot of the time I suspect that money has been handed out to people to boost up the game. Dragon Age 2 comes to mind, it was a piece of s**t(or let's say mediocre game) and yet in the first couple of weeks it was all 8's and 9's for it when it was thoroughly mediocre. Not to mention that some reviewers actually troll in their own way, by giving 5-6 to otherwise excellent games(that would genuinely warrant at least a 8) in order to get hits on websites.

When a magazine/reviewer gives a 10 it loses all credibility with me. 10 would mean that the game is perfect and NO SUCH THING EXISTS.

Look at some of the examples:

DE:HR has four 10 mark critic reviews on metacritic and while I enjoyed it thoroughly(even bough the DLC) it is by no means a 10 worthy game. 85-90 maybe but 100? No freaking way.

Dragon Age 2, which is a mediocre game through and through, got 100 on the Escapist, a mark of shame for this website.

Another quality game(in the high 80's-90 mark to be sure), Uncharted 3, got 19 critic 100 mark reviews.

And people still expect us to take this s**t seriously?

 

[TheDooD]

The reviewing system is using the shitty 69-100 GRADING scale as a reference and its hurting everybody in the long run... This grading curve we been using for far too long. Where everything below a 70-60 is considered "failing". Yet common sense and math are basically taken out when most people see a 70 they think its crap but out of 100 it's pretty fucking high.

 

[Treblaine]

Who do you think is to blame for the score 'inflation' that we see? Gamers, desperately crying out for validation that their chosen game is AMAZING? Reviewers, hoping to drive up traffic either via nerd-baiting or simply by giving a positive review? Developers/Publishers in an attempt to get their game more positive press in the hopes that they sell more copies? All of the above?

has the average review score REALLY been inflated by fan-wankery of the internet:

http://www.metacritic.com/game/nintendo-64/the-legend-of-zelda-ocarina-of-time 9.9 aggregate, from 1998

http://www.metacritic.com/game/nintendo-64/perfect-dark
http://www.metacritic.com/game/nintendo-64/goldeneye-007
http://www.metacritic.com/game/xbox/halo-combat-evolved
http://www.metacritic.com/game/gamecube/metroid-prime

Super high scores there, but more poignantly, what games also get relatively high scores for what are arguably "average":

http://www.metacritic.com/game/nintendo-64/turok-2-seeds-of-evil
http://www.metacritic.com/game/nintendo-64/pilotwings-64
http://www.metacritic.com/game/nintendo-64/007-the-world-is-not-enough

Stupendously high scores did not happen when everyone got broadband, and average games did not get 7-8 because of that either these are all from the 1990's and early 2000's, there was not "traffic" on the internet back then. That is precisely why the internet bubble burst because the market WAS NOT THERE! Only a few had the internet or regularly used it, magazine review ruled the roost and you bought one per week if you were lucky.

Metacritic didn't even exist back then, people had no easy way of knowing what the aggregate review score was.

Face it, such highly weighted because IT MAKES SENSE!

An average game should score in 3/4 of criteria! This makes sense to critics, and to those with a good education in mathematics. The only possible reason for demanding a "5 must be average" is a misguided attempt at symmetry, that "average must be middle".

 

[Deludedfool]

I'd say its pretty busted, why on earth should the average be on the upper limit it just makes judging games harder than it needs to be, anything lower than 6 would never be considered by most people so essentially we have a scale of 1-5 with 1 being 5/10.

I'm not a massive fan of using numbers overall for ratings anyway, when they give individual areas a number i can just about manage but giving an overall game a numbers pointless especially when so many things these days score 9s or 10s.

I feel this is because game reviewers standards don't seem to be progressing as fast as the quality of high end games do. Anything with graphics that don't look like they were with for PS2 will score at least an 8 these days.

 

[babinro]

Personally I'd grade on a 100 scale.

I'd make it clear that the games are being graded based on how they compare against their competitors in the current gaming generation. By using launch titles as a benchmark score it becomes easier to weigh how much a title has improved over the consoles life-cycle.

I'd focus scores entirely on fun factor and the perceived replay value. Things like sound, innovation, graphics, presentation are fine, but they should never factor into a number on the review score.

Is game reviewing 'broken'? I'd say no. It's broken when you get into game comparisons based on score. It works relatively well when telling you whether or not a game is worth your money. I'd give Uncharted 3 an 8/10 where IGN gives a 10/10. In the end, this is a game that's worth your money and both reviews should reflect that.

 

[LavaLampBamboo]

I've always been a little confused by this topic.

Okay so the fans feel like anything below say, I don't know, a 7 is a waste of time. But then again, so do the critics. So do the devs and publishers. Everyone FEELS that a seven is avarage. So why should it matter. The score scale has just been shifted a tad to the right.

It also winds me up when devs talk about the whole idea of people thinking below a 7 is poor. That's not the point. You should be aiming for a 10. If you aren't then I don't think you have any reason to complain about the review system.

Personally, I prefer an out of five scale. It's similar, but it allows less distinction between levels, so it's a slightly fairer playing field.

 

[BlindedHunter]

I think, while there are a lot of good points above about subjectivity and the importance of much more nuanced approaches, that one of the issues to consider with the scoring system is that there is no real definition for average - at least not one that is moving with the times. The reason things like IQ and grading scales "work" is that they maintain an average through constant monitoring and grading curves, but you can't really do that turning such a nuanced thing like a game into numbers. So where something like the newest bland first person shooter might deserve a 5/10 because it really is just another release in a line of games with no big surprises, it isn't what we would call a bad game and, compared to older games that you might call an average it is beautiful.

Basically my point is that we may still be using the same scale we used back in the day, but now what commonly comes out tends to easily surpass the old average.

 

[Treblaine]

I think there is something broken/corrupt in it when almost every AAA game rating nowadays is from 8.5 upwards. No doubt, some of them have high enough production values to ensure a high quality game but a lot of the time I suspect that money has been handed out to people to boost up the game. Dragon Age 2 comes to mind, it was a piece of s**t(or let's say mediocre game) and yet in the first couple of weeks it was all 8's and 9's for it when it was thoroughly mediocre. Not to mention that some reviewers actually troll in their own way, by giving 5-6 to otherwise excellent games(that would genuinely warrant at least a 8) in order to get hits on websites.

When a magazine/reviewer gives a 10 it loses all credibility with me. 10 would mean that the game is perfect and NO SUCH THING EXISTS.

Look at some of the examples:

DE:HR has four 10 mark critic reviews on metacritic and while I enjoyed it thoroughly(even bough the DLC) it is by no means a 10 worthy game. 85-90 maybe but 100? No freaking way.

Dragon Age 2, which is a mediocre game through and through, got 100 on the Escapist, a mark of shame for this website.

Another quality game(in the high 80's-90 mark to be sure), Uncharted 3, got 19 critic 100 mark reviews.

And people still expect us to take this s**t seriously?

*facepalm at poor understanding of maths*

You know for every critic that "over-rates" a game there will on average the same number (or of equivalent "weight") who "under-rate" it?

If you get rid of the over-rating and keep the under-rating, then the actual average slips BELOW where the actual centre of opinion lies! And you can't just arbitrarily say that those in the middle have the "ideal" score, this is supposed to be objective! Do you not understand consensus through aggregate?

It's not the job of critics to be as much like everyone else as possible, it is their job to give their valued opinion and analysis. Yes, some may get carried away, but just as likely some will form an abnormally low opinion. The aggregate is drawn from the CONSENSUS, gathering together and weighing all the scores.

 

[RickyRich]

Personally, I think it should be like schools grade papers . 9.0-10.0 = A(near perfect), 8.0-8.9= B(Very Good), 7.0-7.9= C(Good), 6.0-6.9= D(Okay), 5.0-5.9= F(Not good), <4.9= Why does this exist?

 

[weker]

Scores are the biggest blight in overall media, you can attribute a score, let alone a numerical data, as the scoring is too subjective. If a numerical score is attributed to a game either a set model must be proposed by the reviewer like Jim Sterling. Too many reviewers don't explain how they rate their games, meaning the score they give means NOTHING, it doesn't show anything: is it their enjoyment, quality analysis, emotional impact, innovation, artistic potential.

For example I currently write reviews, the way I currently write reviews, I attempt to remain fairly objective with occasionally a shift due to my own judgment and preference on what is good or bad, such as bad controls. If I had my own way with my reviews I would score them based on my own personal enjoyment of the game, as this provides an 100% accurate reviews as it is MY OWN PERSONAL ENJOYMENT, while with the current model it is my opinion on what I think the quality is, which has a more objective tone.

I also would rate games 1-5 (as I have to score them *rolls eyes) as it would suggest the most basic thoughts towards the game, in other words bad, fairly bad, average, good, and great, while currently with the 1-10 scale gamers throw hissy fits over if a game is a 10 or a 9.
Don't get me started on the personal hell that decimal point scores cause me.

 

[AntiChri5]

I think that the greatest problem with reviewing games is how quickly reviews must be delivered. Most reviews come out on or before release, and the longer after the game is released the less relevant it is. With a late review, most people interested have already made a decision and there are already many reviews to compete with.

How much interest would a new Mass Effect 2 review get today? Probably less then 10% of the interest it would have recieved on the day the game was launched.

While you can accurately get an impression of a CoD game in the short span between getting your review copy from the publishers and the game being released to the general public, you can't do that for a game like Morrowind or New Vegas. I absolutely hated my first five hours of Morrowind, but then it went on to be one of my favouritest games ever. Some games just take longer to judge, you can only get an impression of a fraction of the game in a short span of time. A day or two is probably long enough to get an accurate impression of Morrowinds melee combat, but not if you want an accurate impression of the magic and stealth systems.

 

[Kukakkau]

Yes it is - that's why if someone askes me my view on a game i'll break into positives and negatives and let them decide

 

[count9]

It's kinda funny, it used to be magazines can say: "Don't pirate games, read our reviews and then you can decide which games are worth your time. You don't have to pirate them to try them out." Now you can't fucking tell at all, I can't even remember the last time a score under 7 had a meaning. I mean, if a game got a 3/10 and another one got 5/10, does it mean anything to you? It could've been reviewer bias for all I know, when a full 2 points on a 10 point scale could be margin of error the rating system holds no weight.

 

[Treblaine]

Actually, systems such as reviews on scale of 100 (they are all scales of 100 basically, even on a 1-10 scale because they use to the tenths decimal place) tend to be totally subjective. The metric is, in and of itself, flawed. Leonard Mlodinow explains it beautifully in the book, The Drunkard's Walk. Essentially, people have tried to create a system such as this for grading papers for school. This one fella' took months drilling strict criteria for his out of 100 scale paper grading system. He trained 25 people. After the training, he gave them all the same stack of 25 papers to grade. When he looked at the results of his work, peoples grades varied by as much as 12 points on the same paper. He also tells the story of how two of his son's friends turned in identical papers, accidentally. Instead of getting caught, one received a 90 while the other received a 79. For the number system to mean anything, there would have to be a palpable difference from one point to the next, but there isn't.

The margin for error is huge. I would invite you to check out metacritic (I know, I hate it too, but it shows my point beautifully). In the official reviews column on a game that has seen international release, there tends to be a huge difference in scores out of 100. One game (it's been a while so I don't remember the game) had a top score of 71, while the lowest score on the list for the game was a 10.

Also, I know no one is a fan of a by the numbers system.

But isn't the point of Metacritic to smooth out that margin of error?

Like how a junk rifle can be inaccurate, keep firing it at the target and the centre of the cluster of bullet holes will be actually your centre of aim.

You give the perfect example with your paper marking to the inaccuracy between two shots of a rifle, the identical paper marked by different people, to get the true score you have it mark it by many more teachers and the most accurate score is the average of all their scores.

Metascore. THAT is what we are talking about here. Frankly, critics might as well never reveal publicly their score to a game, they should send it straight to metacritic to find the aggregate. As a single score in itself is useless due to the inaccuracies in trying to quantify ones judgement. And it doesn't do any good as then the fanboys and haters say "GRRR, IGN skewed the result, if it wasn't for them this game would have had a different score! GRRR"

The worth of a single critic should be in their prose. What they actually write about a given work, that is the most important guide to the customer.

As to Modern Warfare 3, consider this: it may be hardly an improvement over COD4 but:
-COD4 is still a good game, 4 years later
-No other game has really surpassed it in what it does.

So, standards have NOT gone up significantly, MW3 is a bit better than COD4 in the most valued areas and enough things are changed around for it to get the same score as COD4. They may rate it a bit higher, but another critic will rate it a bit lower. THERE IS NO REASON FOR CONFORMITY! People can have varying opinions and judgements.

 

[jacilon]

We should raise the max score to one million.

 

[masticina]

Mmm it is not easy I do agree that it seems like the gamereview industry has been pulling up allot.

Self? Well Self I am way thougher on games then some review websites. Of course we know the japanese "Oh it is a final fantasy 10/10" and the "The new Halo is definitely one of the best experiences this year" of the western market.
Pay
But some websites actually dare to be negative. Giantbomb for instance it does gives at times 2 out of 5 stars "Meh" and yes also 4 our of five stars.

Reviewing is subjective anyway and I would agree that a 10/10 is possible even for a game with flaws. IF those flaws are inconsequential compared to what works. If there are 3-4 flaws on 250 things done well. You know then it is a 10!

In the end I read reviews.. yeah I read them, I watch gameplay videos I make up my own mind. Is Halo a great shooter? Probably but I don't do shooters so I could care less.

Do I find that bringing out a new title every year with only small progression is cheap? Yes! Paying full price for a few small updates or addition of a few more maps ... seriously

So in the end I will say that I rather read reviews then go on by their numbers. I mean I have enough games that we're "panned" by giving a 7.5. I as player can look through those flaws and enjoy what works.

Take The Saboteur not the highest scoring game, neither without flaws, but it is fun! Very Very Fun!

 

[Daveman]

I dunno, I take 9-10 as an instant classic, 7-8 as good regardless, 5-6 as good if you enjoy that genre and anything less as pretty poor.