260: 1984 Out of 10

[deth2munkies]

I do agree that score inflation is quite rampant among most mainstream magazines/sites like PC Gamer, EGM, Gamespot, and the like, but I'd really like to touch on the last point: that score inflation is partially due to publishers imposing their wills upon game critics.

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

When advertising dollars get involved, it becomes a matter of the critic's integrity versus cashflow. The big example that you cited was the Gamespot Kane and Lynch scenario which does illustrate something very wrong with the way Gamespot does business. No reviewer should ever be fired because he did his job. His job is to criticize the game, separate the good from bad, and give an overall impression to the consumer. As long as he remains consistent in his feedback, there is no reason he should lose his job, regardless of what developers or fanboys might say.

 

[Kukakkau]

You know, the Norwegian system for reviewing everything actually works pretty well. It is a scale from one to six, no decimals. I don't know if it is due to the size of the scale or more to the culture around it, but it is commonly accepted that the scale is:

1 = horrible
2 = bad
3 = mediocre
4 = Good
5 = Great
6 = Mindblowing

I like this - simple and effective

But I'd rather reviews didn't give number scores and just described their opinion on the game/movie whatever with the positives and the negatives

Example:
+great acting
+well directed

-bad effects
-poor story

 

[WaderiAAA]

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

I disagree with this. Why? Because a reviewer is supposed to give the consumer an idea of how much they'll enjoy the game, and even if a company listed their arguments to consumers as well as to reviewers, 99 out of 100 consumers wouldn't bother to read it. The game ought to speak for itself. If the game really is as good as the publisher would argue, than the reviewer should be able to list those exact positive points without having to read the letter first. If the reviewer isn't able to do that, then those points aren't valid.

I also wonder. Would it be okay if a reviewer had decided to give a game a 8.0 after playing it, but then read the letter and thought "hm, that is a good point" and gave it 8.5 instead? I mean, to me, if they changed the score at all due to the letter, then the score would no longer accurately display how much the reviewer enjoyed it.

 

[deth2munkies]

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

I disagree with this. Why? Because a reviewer is supposed to give the consumer an idea of how much they'll enjoy the game, and even if a company listed their arguments to consumers as well as to reviewers, 99 out of 100 consumers wouldn't bother to read it. The game ought to speak for itself. If the game really is as good as the publisher would argue, than the reviewer should be able to list those exact positive points without having to read the letter first. If the reviewer isn't able to do that, then those points aren't valid.

I also wonder. Would it be okay if a reviewer had decided to give a game a 8.0 after playing it, but then read the letter and thought "hm, that is a good point" and gave it 8.5 instead? I mean, to me, if they changed the score at all due to the letter, then the score would no longer accurately display how much the reviewer enjoyed it.

 

[WaderiAAA]

I personally think that it is the job of every game company to do everything possible to put their games in the hands of consumers. If they write an e-mail requesting their game be given a positive score and list the merits, that's perfectly fine. What they're doing is making an argument towards one conclusion and being an advocate for their product. What is wrong is when game critics rely on the publisher's views instead of giving their own opinion. For instance, there's nothing wrong with saying, "Oblivion has a vast, open world that literally takes hours to explore," even though that might be right on the box, it's a truth and it's my opinion as well as that of the game publishers.

I disagree with this. Why? Because a reviewer is supposed to give the consumer an idea of how much they'll enjoy the game, and even if a company listed their arguments to consumers as well as to reviewers, 99 out of 100 consumers wouldn't bother to read it. The game ought to speak for itself. If the game really is as good as the publisher would argue, than the reviewer should be able to list those exact positive points without having to read the letter first. If the reviewer isn't able to do that, then those points aren't valid.

I also wonder. Would it be okay if a reviewer had decided to give a game a 8.0 after playing it, but then read the letter and thought "hm, that is a good point" and gave it 8.5 instead? I mean, to me, if they changed the score at all due to the letter, then the score would no longer accurately display how much the reviewer enjoyed it.

I get that argument, but where does the line go. I think they shouldn't take the letter into account at all. If you agree with me on that, what would the point be in sending the letter in the first place?

 

[PurpleGoatMan]

As people like karhell and Frybird say, the current numerical scoring for games is almost completely useless! The only use it has is to see how the reviewer thought, it was; your personal opinion may be changed or altered just by reading the review before even buying the game, as the article points out.

For example a review may state that the story line was "unrealistic, uninteresting and dull." but your individual perspective will only be truly made up after playing or viewing the gameplay.

As this amazingly concise and precise article states: the hype that the Press and the reviewers give these games are also daftly absurd. What we need is proper reviews that are based on the individual appeal to a stereotyped group of people (oxymoronic and arguement invoking statement I know...) That people can then make their mind up.

Or perhaps, if people knew the reviewer's personal opinion boundaries; e.g. what quantifies a "bad plot" then that would be okay, as long as the reviewer was truthful and honest in his opinion and he wasn't just being abusingly pressured into providing a biased review, which we all know would never happen...

What a world we live in, eh?

At least we have the Escapist!

Anyway, I thought the article hit the nail on the head there, and I completely agree with it.

 

[Thedutchjelle]

I don't really trust any scores/reviews at all anymore. I'm usually just going by my instinct and opinions of people I know. This has served me okay-ish so far.

 

[Harkwell]

I have to say, at the end of the day, it comes down to me, $5, and a rental store. I never completely trust large game reviewers. Its too easy for them to slap on a good score and call it a day. That being said I can't name any reliable reviewers. It seems everyone is shooting for the top to become big. Once they become big, they become complacent. Once they become complacent, they lose quality.

 

[Georgie_Leech]

I've given up on reviews in general. I listen more to close friends on how much they enjoy a game, and if they do, I'll rent/borrow it for a day or so to try it. Orwell was right in that reviewers become more and more slanted to the top end of the spectrum (I've lost track of how many "Movies of the year" have been developed). He also has good ideas as to how the problems may be fixed. The problem is that the pressures will always be there. Should a reviewer become known for honesty, they will become more popular. As they become more popular, developers/PR people will lean on them more heavily. Thus, he/she will find themselves slipping to the inflated opinions.

 

[veloper]

YES. Very much yes. The situation has also made it hard to differentiate score-wise between a game in which the reveiwer simply unable to find enough wrong with to slap the "7/10" sticker on them without strange looks from his editor and a game which was extreamly decent but had a few bug and enough "Objectively" wrong with it to metit a downgrade to a 7/10. The reviewer may have found it an allround more inspired and better game albeit a little more rough but 7/10 is a 'safe' option.

Infact even 6s are getting more and more rare. and a common 5 or 4 is unheard of.

I never saw what the big deal was.

Just aslong as you're aware of it, you can translate 7 as mediocre, 8 as average, 9 as good, etc.
If this scheme fools the publisher, all the better. WE know.

The score inflation isn't a problem. The problem is lousy or dishonest reviewers and nothing will fix that.

 

[veloper]

YES. Very much yes. The situation has also made it hard to differentiate score-wise between a game in which the reveiwer simply unable to find enough wrong with to slap the "7/10" sticker on them without strange looks from his editor and a game which was extreamly decent but had a few bug and enough "Objectively" wrong with it to metit a downgrade to a 7/10. The reviewer may have found it an allround more inspired and better game albeit a little more rough but 7/10 is a 'safe' option.

Infact even 6s are getting more and more rare. and a common 5 or 4 is unheard of.

I never saw what the big deal was.

Just aslong as you're aware of it, you can translate 7 as mediocre, 8 as average, 9 as good, etc.
If this scheme fools the publisher, all the better. WE know.

The score inflation isn't a problem. The problem is lousy or dishonest reviewers and nothing will fix that.

Problem is, when is a 7 "a good 7" "A mediocore 7" or "I couldn't be aresed so here is a 7"

The consumer is left with a meanless score. A game like FEAR 2 does deserve a 5/10 because it is the very definition of bland and unremarkable. But because there was nothing game-breaking it got mainly 7s and 8s. Now take for example a more flawed but more abitious games like STALKER; Clear ~Sky. Utterly broken upon release but a great game in there that gets marked down for techincal issues to a 7/10.

some games are good but just don't engage enough or are two flawed/ held back to get a higher score than 7 BUT there is a whole raft of frankly boring shit that is stamped with the 7/10 because it runs nicely but nothing else.

They have decimals for that. Consider a star system like it's being used here on the Escapist is about as useful as a 7 to 10 scale. The score alone won't allow for much comparison, because one star covers alot.

An honest and diligent reviewer if one exists, can in theory rank games in the same genre by score, with decimals (like 7.1 < 7.2). It would still be an opinion, but it would be quantifiable again.

 

[IronCladNinja]

Good article, I'd have to agree with a lot of it points. Personally I prefer to the avoid the whole problem myself by sticking to known variables like Valve, or Nintendo, which consistently make good games. Versus EA, which pushes a lot of average. If I do read reviews, I read multiple for the same game, and glean what useful information I can.

Interestingly enough, I find 1984 to be the single most over-hyped novel ever written. Sure it's a great book, but Animal Farm is pretty much the exact same thing, and I personally like it better. It is relevant yes, but these days people are way to deep into this whole "communism/socialism" thing. If you want to read a REALLY relevant novel, read "Brave New World" by Aldous Huxley, he pretty much predicts the future, then parodies it.

 

[PlasticTree]

At the moment I'm too lazy to actually comment anything useful about this topic, but I nonetheless want to say the following: nice article, keep it up.

Yeah, that's all.

 

[KingArmery]

There used to be a good site called Crispy Gamer (they fired all of their staff and hired new people, so I don't know if the site is worth anything anymore) where the journalists did three things that can very directly help this problem:

1) they ditched the out of 10 rating system and instead assigned "buy it, try it, or fry it" to the games.

2) for very hyped up games that got high scores (Call of Duty Modern Warfare 2, for example), they would take all of the negatives that had been pointed out by other reviewers into one meta-review.

3) the journalists had accounts and were active in the forums and comments sections so they could explain their scores and hear what people had to say about what they reviewed.

It kept the journalists in check, both on and off the site, and also gave the reviewers wiggle-room with their scores because they would not have to worry about offering a 9/10 for a 7/10 game because they gave a worse game a better score than that.

 

[Tharticus]

Good article. I like that how you relate to George's book 1984.

I have given up on every gaming site such as GameSpot, IGN and after that, published magazines like Game Informer and the like. Then I came to this site and I like it a lot.

Two incidents that I would like to call out that made me change my attitude: Jeff Gerstmann with his review of Kane & Lynch and Dan Hsu's article on editorial integrity.

I'm very critical that people only read the scores and not bother with the writers opinion on why they should buy this game or why they shouldn't buy this game. Jeff Gerstmann call it out that Kane & Lynch game isn't a good game but Gamespot and Eidos has advertising of the game on Gamespot's page. It's almost the same way giving a game a very low score like Fight Club (which has little to do with the novel or movie) and have a advertising page to the next. Few days later, Mr. Gerstmann was fired from GameSpot and went to make a video game site called Giant Bomb.

Possibly the straw that broke the camel's back would happen to be Dan Hsu's article on Editorial Integrity. [http://www.1up.com/do/blogEntry?bId=6228583&publicUserId=5379799]
Dan Hsu was one of the harshest but very insightful video game journalists I have ever seen. If anyone recalled, he interviewed Peter Moore on the Xbox 360 and brutalized Peter Moore for the faults that the Xbox 360 has gone through. Many people consider his questioning to be inappropriately rude, confrontational, or aggressive but I find that appropriate. Dan actually went in there and asked the sorts of questions you see every day online, in the indecorous language of the hardcore forum. Days later, he left EGM and started Bitmob.

But the gigantic issue of video games these days is, why gaming reviews from certain sites and magazines are reviewing games that is stuck to only adolescent boys?

I quote Rob Fahey [http://www.gamesindustry.biz/articles/critical-failure-editorial] from GamesIndustry.biz

Talking to the creators of kids' games about game journalism, for example, is usually a depressing affair. While developers are rarely terribly enamored of writers in the first place - after all, you can't expect hugs and kisses from the people who create the products over which you've set yourself up as an arbiter of quality - those who work on kids' games are most often genuinely bitter, angry, or both.

At times, maybe the gaming journalism is broken. As long as video games are run by publishers and video game journalists plays nice to each other, the system continues to be broken. So some writers are afraid to ask the tough questions, or to criticize what should be criticized, because they're afraid of backlash from the companies from a support standpoint, from an advertising standpoint or worse, from their own editors who don't want to piss anyone off.

Then again, the entire world has their own opinions and differences. Some games are overhyped and people buy it from the word of mouth. Who knows, maybe video game reviews doesn't lead to massive game sales.

 

[poiuppx]

Meh. The only review that matters is your own. Hence why I tend to set a kind of mental price-point for games in my mind; if I had trust in the company/studio and the game is of a genre I like, or in a series I like, it's a buy if nothing else to show I have faith in said company. Beyond that, it's a matter of how low the price goes versus how much fun I think I'll have with the title. If the value of the game reaches below the pricepoint for comparitive value if I, say, bought a DVD or something comparable with the money, then it's a good deal.

 

[ranger19]

I've been overcome by a bout of sleepyness and couldn't make my way through all the comments, but I wonder what the writer would think about the actual quality of games. As gaming becomes a better established medium, developers get more experience, and production costs skyrocket, I think it's very plausible that average game quality is increasing. This certainly does not answer the whole question, but is worth mentioning I think. Because if critics' expectations stay the same, over time this sort of inflation will be inevitable. Should critics be ever raising their expectations?

 

[redsoxfantom]

I personally rearely take reviews seriously. Unless they're reviews for terrible games. Those are HILARIOUS.