Is game reviewing 'broken' as a system?

[Treblaine]

I think it should change, I'm tired of seeing 7-8 be an 'average' score in a ten point scale.

Why?

Why assume that "average" must be "middle". That's just mathematically ignorant.

Review scores are like scores YOU would get doing a test/exam in school. The average person should not get HALF the questions wrong. A game should not fail on HALF the criteria! The average should be as close to 10 as possible! As just like every student should be answering every question right, every game should be succeeding in every criteria.

Would you accept being treated by an "average" doctor who couldn't correctly answer HALF OF THE QUESTIONS POSED TO THEM!! That's not an average doctor, that's really bad doctor.

When reviewing things including games, 10/10 should be for a game that is more of less, for this time and price, for its genre and type it is as good as it be. Temporal perfection.

There is nothing wrong with 7.5 being average.

It is asinine and pointless to say "oooh, we have to use the WHOLE point scale" there is no reason to.

 

[Robert Ewing]

It's not broken by any means. It does exactly what it sets out to do. It gives a 'professional' yet 'personal' opinion on a game, and judges it by his or her standards.

If you understand his or her standards, or compare it to other reviews, then you get a very good idea as to the quality of the game.

Reviewing is useful, and why not review stuff?

 

[hooksashands]

I used to think nothing of it. Percentages, stars, 1/10. It felt as natural as the period at the end of a sentence, but now it's become more of a redundant summary. Our own site kicks reviews to individual staff, who don't pretend to be unbiased--the game is being looked at through only one set of eyes, I've never seen them presume others are going to agree with them--they simply offer their version of how they found the game's playability.

No, it isn't broken. The beauty of the so-called system is that there is not just one review for every game. There is a multitude, and it is therefore unbiased. Whoever made that thread yesterday complaining about Modern Warfare 3's Metacritic score fails to grasp this simple concept: There is no undisputed authority that can chisel a review for a game into stone, never to be questioned. The thought police will not come to your house and arrest you for liking something everyone else hates or hating something everyone else likes.

What is this paranoid fantasy that review scores on every site are enforced by some quota men working from the shadows? This assumes the impossible: That reviewers conspire with each other somehow. That they convene and say "Okay guys, here is the model for our review of Skyrim. Now of course each of us is just one person, but we have to make it look like we got everyone's opinion on-board so we'll all post roughly the same scores so we can fool 'em into thinking we're unbiased." No, a single review will always be biased, but if you look at several from your perspective, then it filters out the bias and leaves you with an idea of what everyone thought. The next best thing is to just play the game yourself and decide.

 

[Denariax]

This is kinda funny. But here's the bottom line; Popular games are the ones that have big name developers. In the end, 'reviewers' will stamp it with golden money piles to think they get something in return, for example viewers, which will raise their revenue. The problem with this is that all of the world's population is comprised of self-righteous gits; they will never 'award' anything they don't like, or; dare I say it; will only award popular games for previously stated income. This includes myself, except I don't have a website.

I'm an autistic, schizophrenic, bipolar, sociopathic mess of a human being; so why am I the one always trying to think for others?

 

[surg3n]

Yes, definitely. I am far more likely to judge a game based on youtube videos, escapist videos, yogscast, and forum posts - which I guess is more like a community review. I just don't trust reviewers to give an unbiased opinion. At the end of the day, a review is just a well written opinion (arguably!) - these days we can summarise several opinions and get opinions from people whose opinion we might actually care about. Never mind which reviewer got paid to say this or that, I am more interested in how much Yahtzee hated on it, or I'll watch the yognaughts do a play through, and get a real understanding of the game.

Times have changed, it's not like we have to go and spend our pocket money on a games magazine to get an idea about new games - the internet has levelled the playing field, and reviewers will have to adapt... the greasing of palms will become pointless once more people realise just how corrupt reviewers are forced to be a lot of the time.

 

[GigaHz]

I've also noticed the change in scoring trends and I agree with OP. There is no point spread anymore.

Literally, any half decent game will get some variation of a 9, or a high 8 at the very least while anything excellent will get a 9.5 or above.

What is the point of rating just about every other game a 9?

If there are in fact a surplus of 9s, then scoring standards need to change to reflect that. Games that are now scoring in the 8-8.9 range should be 5s, games rated 9.0-9.5 changed to 6's to 8's, and lastly games rated 9.5-10s changed to 9s and 10s.

Obviously anything below that would be in the 1-4 range.

But who am I kidding? Something like this will never happen. Kids these days go into nerd rage when their favourite game doesn't achieve a 9. Would you imagine if they had to *gulp* consider purchasing something that might be a 6 or 7? Oh the shame!

 

[predatorpulse7]

I think there is something broken/corrupt in it when almost every AAA game rating nowadays is from 8.5 upwards. No doubt, some of them have high enough production values to ensure a high quality game but a lot of the time I suspect that money has been handed out to people to boost up the game. Dragon Age 2 comes to mind, it was a piece of s**t(or let's say mediocre game) and yet in the first couple of weeks it was all 8's and 9's for it when it was thoroughly mediocre. Not to mention that some reviewers actually troll in their own way, by giving 5-6 to otherwise excellent games(that would genuinely warrant at least a 8) in order to get hits on websites.

When a magazine/reviewer gives a 10 it loses all credibility with me. 10 would mean that the game is perfect and NO SUCH THING EXISTS.

Look at some of the examples:

DE:HR has four 10 mark critic reviews on metacritic and while I enjoyed it thoroughly(even bough the DLC) it is by no means a 10 worthy game. 85-90 maybe but 100? No freaking way.

Dragon Age 2, which is a mediocre game through and through, got 100 on the Escapist, a mark of shame for this website.

Another quality game(in the high 80's-90 mark to be sure), Uncharted 3, got 19 critic 100 mark reviews.

And people still expect us to take this s**t seriously?

 

[TheDooD]

The reviewing system is using the shitty 69-100 GRADING scale as a reference and its hurting everybody in the long run... This grading curve we been using for far too long. Where everything below a 70-60 is considered "failing". Yet common sense and math are basically taken out when most people see a 70 they think its crap but out of 100 it's pretty fucking high.

 

[Treblaine]

Who do you think is to blame for the score 'inflation' that we see? Gamers, desperately crying out for validation that their chosen game is AMAZING? Reviewers, hoping to drive up traffic either via nerd-baiting or simply by giving a positive review? Developers/Publishers in an attempt to get their game more positive press in the hopes that they sell more copies? All of the above?

has the average review score REALLY been inflated by fan-wankery of the internet:

http://www.metacritic.com/game/nintendo-64/the-legend-of-zelda-ocarina-of-time 9.9 aggregate, from 1998

http://www.metacritic.com/game/nintendo-64/perfect-dark
http://www.metacritic.com/game/nintendo-64/goldeneye-007
http://www.metacritic.com/game/xbox/halo-combat-evolved
http://www.metacritic.com/game/gamecube/metroid-prime

Super high scores there, but more poignantly, what games also get relatively high scores for what are arguably "average":

http://www.metacritic.com/game/nintendo-64/turok-2-seeds-of-evil
http://www.metacritic.com/game/nintendo-64/pilotwings-64
http://www.metacritic.com/game/nintendo-64/007-the-world-is-not-enough

Stupendously high scores did not happen when everyone got broadband, and average games did not get 7-8 because of that either these are all from the 1990's and early 2000's, there was not "traffic" on the internet back then. That is precisely why the internet bubble burst because the market WAS NOT THERE! Only a few had the internet or regularly used it, magazine review ruled the roost and you bought one per week if you were lucky.

Metacritic didn't even exist back then, people had no easy way of knowing what the aggregate review score was.

Face it, such highly weighted because IT MAKES SENSE!

An average game should score in 3/4 of criteria! This makes sense to critics, and to those with a good education in mathematics. The only possible reason for demanding a "5 must be average" is a misguided attempt at symmetry, that "average must be middle".

 

[Deludedfool]

I'd say its pretty busted, why on earth should the average be on the upper limit it just makes judging games harder than it needs to be, anything lower than 6 would never be considered by most people so essentially we have a scale of 1-5 with 1 being 5/10.

I'm not a massive fan of using numbers overall for ratings anyway, when they give individual areas a number i can just about manage but giving an overall game a numbers pointless especially when so many things these days score 9s or 10s.

I feel this is because game reviewers standards don't seem to be progressing as fast as the quality of high end games do. Anything with graphics that don't look like they were with for PS2 will score at least an 8 these days.

 

[babinro]

Personally I'd grade on a 100 scale.

I'd make it clear that the games are being graded based on how they compare against their competitors in the current gaming generation. By using launch titles as a benchmark score it becomes easier to weigh how much a title has improved over the consoles life-cycle.

I'd focus scores entirely on fun factor and the perceived replay value. Things like sound, innovation, graphics, presentation are fine, but they should never factor into a number on the review score.

Is game reviewing 'broken'? I'd say no. It's broken when you get into game comparisons based on score. It works relatively well when telling you whether or not a game is worth your money. I'd give Uncharted 3 an 8/10 where IGN gives a 10/10. In the end, this is a game that's worth your money and both reviews should reflect that.

 

[LavaLampBamboo]

I've always been a little confused by this topic.

Okay so the fans feel like anything below say, I don't know, a 7 is a waste of time. But then again, so do the critics. So do the devs and publishers. Everyone FEELS that a seven is avarage. So why should it matter. The score scale has just been shifted a tad to the right.

It also winds me up when devs talk about the whole idea of people thinking below a 7 is poor. That's not the point. You should be aiming for a 10. If you aren't then I don't think you have any reason to complain about the review system.

Personally, I prefer an out of five scale. It's similar, but it allows less distinction between levels, so it's a slightly fairer playing field.

 

[BlindedHunter]

I think, while there are a lot of good points above about subjectivity and the importance of much more nuanced approaches, that one of the issues to consider with the scoring system is that there is no real definition for average - at least not one that is moving with the times. The reason things like IQ and grading scales "work" is that they maintain an average through constant monitoring and grading curves, but you can't really do that turning such a nuanced thing like a game into numbers. So where something like the newest bland first person shooter might deserve a 5/10 because it really is just another release in a line of games with no big surprises, it isn't what we would call a bad game and, compared to older games that you might call an average it is beautiful.

Basically my point is that we may still be using the same scale we used back in the day, but now what commonly comes out tends to easily surpass the old average.

 

[Treblaine]

I think there is something broken/corrupt in it when almost every AAA game rating nowadays is from 8.5 upwards. No doubt, some of them have high enough production values to ensure a high quality game but a lot of the time I suspect that money has been handed out to people to boost up the game. Dragon Age 2 comes to mind, it was a piece of s**t(or let's say mediocre game) and yet in the first couple of weeks it was all 8's and 9's for it when it was thoroughly mediocre. Not to mention that some reviewers actually troll in their own way, by giving 5-6 to otherwise excellent games(that would genuinely warrant at least a 8) in order to get hits on websites.

When a magazine/reviewer gives a 10 it loses all credibility with me. 10 would mean that the game is perfect and NO SUCH THING EXISTS.

Look at some of the examples:

DE:HR has four 10 mark critic reviews on metacritic and while I enjoyed it thoroughly(even bough the DLC) it is by no means a 10 worthy game. 85-90 maybe but 100? No freaking way.

Dragon Age 2, which is a mediocre game through and through, got 100 on the Escapist, a mark of shame for this website.

Another quality game(in the high 80's-90 mark to be sure), Uncharted 3, got 19 critic 100 mark reviews.

And people still expect us to take this s**t seriously?

*facepalm at poor understanding of maths*

You know for every critic that "over-rates" a game there will on average the same number (or of equivalent "weight") who "under-rate" it?

If you get rid of the over-rating and keep the under-rating, then the actual average slips BELOW where the actual centre of opinion lies! And you can't just arbitrarily say that those in the middle have the "ideal" score, this is supposed to be objective! Do you not understand consensus through aggregate?

It's not the job of critics to be as much like everyone else as possible, it is their job to give their valued opinion and analysis. Yes, some may get carried away, but just as likely some will form an abnormally low opinion. The aggregate is drawn from the CONSENSUS, gathering together and weighing all the scores.

 

[RickyRich]

Personally, I think it should be like schools grade papers . 9.0-10.0 = A(near perfect), 8.0-8.9= B(Very Good), 7.0-7.9= C(Good), 6.0-6.9= D(Okay), 5.0-5.9= F(Not good), <4.9= Why does this exist?

 

[weker]

Scores are the biggest blight in overall media, you can attribute a score, let alone a numerical data, as the scoring is too subjective. If a numerical score is attributed to a game either a set model must be proposed by the reviewer like Jim Sterling. Too many reviewers don't explain how they rate their games, meaning the score they give means NOTHING, it doesn't show anything: is it their enjoyment, quality analysis, emotional impact, innovation, artistic potential.

For example I currently write reviews, the way I currently write reviews, I attempt to remain fairly objective with occasionally a shift due to my own judgment and preference on what is good or bad, such as bad controls. If I had my own way with my reviews I would score them based on my own personal enjoyment of the game, as this provides an 100% accurate reviews as it is MY OWN PERSONAL ENJOYMENT, while with the current model it is my opinion on what I think the quality is, which has a more objective tone.

I also would rate games 1-5 (as I have to score them *rolls eyes) as it would suggest the most basic thoughts towards the game, in other words bad, fairly bad, average, good, and great, while currently with the 1-10 scale gamers throw hissy fits over if a game is a 10 or a 9.
Don't get me started on the personal hell that decimal point scores cause me.

 

[AntiChri5]

I think that the greatest problem with reviewing games is how quickly reviews must be delivered. Most reviews come out on or before release, and the longer after the game is released the less relevant it is. With a late review, most people interested have already made a decision and there are already many reviews to compete with.

How much interest would a new Mass Effect 2 review get today? Probably less then 10% of the interest it would have recieved on the day the game was launched.

While you can accurately get an impression of a CoD game in the short span between getting your review copy from the publishers and the game being released to the general public, you can't do that for a game like Morrowind or New Vegas. I absolutely hated my first five hours of Morrowind, but then it went on to be one of my favouritest games ever. Some games just take longer to judge, you can only get an impression of a fraction of the game in a short span of time. A day or two is probably long enough to get an accurate impression of Morrowinds melee combat, but not if you want an accurate impression of the magic and stealth systems.

 

[Kukakkau]

Yes it is - that's why if someone askes me my view on a game i'll break into positives and negatives and let them decide

 

[count9]

It's kinda funny, it used to be magazines can say: "Don't pirate games, read our reviews and then you can decide which games are worth your time. You don't have to pirate them to try them out." Now you can't fucking tell at all, I can't even remember the last time a score under 7 had a meaning. I mean, if a game got a 3/10 and another one got 5/10, does it mean anything to you? It could've been reviewer bias for all I know, when a full 2 points on a 10 point scale could be margin of error the rating system holds no weight.