Analyst: Aggregate Review Scores Mean Nothing

[Greg Tito]

Analyst: Aggregate Review Scores Mean Nothing

Media analyst says that "grade-grubbing" does nothing for the sales of their games.

Media and entertainment analyst, Doug Creutz of the Cowen Group stated that a survey conducted by his company proves that game review scores are the least influential factor when consumers are deciding to buy a game. Among 8 different factors, aggregate scores on sites like Metacritic ranked as least important. The highest factors ranked were genre and whether the gamer enjoyed previous versions of the game, underlining why sequels are so important to game publishers.

Creutz stated that his report was highlighting how unimportant reviews are for a specific reason. "We note this, in part, because of persistent rumors that some game developers have been jawboning game reviewers into giving their games higher critical review scores," Creutz's report said. "We believe the publishers are better served by spending their time on the development process than by 'grade-grubbing' after the fact."

That is quite a tongue-lashing to game publishers, to be sure. With this report coming on the heels of Jesse Divnich stating that marketing is more important than reviews [http://www.escapistmagazine.com/news/view/96177-Analyst-Marketing-Trumps-Reviews-in-Driving-Game-Sales], one wonders if we will see shift in how publisher promote their games.

Or, as Creutz said, they should just concentrate on making a great game instead of worrying about what Jeff Gerstmann [http://en.wikipedia.org/wiki/Jeff_Gerstmann] happens to write about their game.

Source: Gamasutra [http://www.gamasutra.com/view/news/26223/Analyst_Review_Scores_Least_Important_Factor_For_Game_Purchases.php]

Permalink

 

[silentsentinel]

I pay attention to reviews. Not the scores, of course, but the commentaries. Word of mouth and company reputation is also a factor.

 

[squid5580]

Can't we just agree to disagree. Or even say "who knows" since no one really does. You ask a small minority thier opinion and that is all you have. They don't speak for the market. Some are and some aren't. The only way to know for sure is take them down and see what happens.

 

[MR T3D]

well knowing 1/2 of the reviews ARE basically adverts... i'm not surprised.
but i often look at average user rating for game myself.

 

[scotth266]

And that's why I never bought into the concept of reviewing by scores. Stars is a decent system, but it still is flawed... in the end, I just go by a three-tier system: buy it, rent it, or burn it, and I always mention conditions (buy X if you really like the idea of Y, rent it otherwise.)

I feel that my system gives me a better estimation of how people view the relative worth of games.

 

[Outlaw Torn]

One thing that annoys me is when you get into an argument/discussion about games (for example, saying how awful Modern Warfare 2 is) one of the first things defendants come out with is that the game got 9-10/10 in such and such a site/magazine review. So what? I also dislike it when publishers decide to slap review scores and testimonials all over the box. If I don't know much about a game then a nice picture on the back or a list of game features is more likely to win me over than GenericReviews.com saying 'Thrilling and cinematic! Eleventy three out of ten!'.

 

[Izerous]

is more likely to win me over than GenericReviews.com saying 'Thrilling and cinematic! Eleventy three out of ten!'.

Those are not there to win you over, they are there to win over the poor fools such as the gamers parents who are trying to buy a gift.

Little johnny likes videos games this one is rated "Eleventy three out of ten" he should like it.

 

[Doug]

is more likely to win me over than GenericReviews.com saying 'Thrilling and cinematic! Eleventy three out of ten!'.

Those are not there to win you over, they are there to win over the poor fools such as the gamers parents who are trying to buy a gift.

Little johnny likes videos games this one is rated "Eleventy three out of ten" he should like it.

And yet, it seems not to work. Primarily because parents don't read gamer sites unless they are gamers themselves (And if they are, its pretty likely they know how bullshit some reviews are).

 

[Izerous]

is more likely to win me over than GenericReviews.com saying 'Thrilling and cinematic! Eleventy three out of ten!'.

Those are not there to win you over, they are there to win over the poor fools such as the gamers parents who are trying to buy a gift.

Little johnny likes videos games this one is rated "Eleventy three out of ten" he should like it.

And yet, it seems not to work. Primarily because parents don't read gamer sites unless they are gamers themselves (And if they are, its pretty likely they know how bullshit some reviews are).

Unintentionally chopped too much off the Nexus Zef post meant to include the section about it talking about writing the reviews on the box itself.

 

[Therumancer]

Well I don't think metacritic scores mean much, but in general I think specific well trafficed review sites are, after all there is a lot of noise about the review scores both before and right after release of a game. I also believe that when reviewers become successful their success leads to increasing pressure from game publishers who buy ad space at the very least.

I've also noticed for example that pretty much all reviews are top heavy. An average game usually nailing about an 8 out of 10 or so as opposed to a 5 (an actual average score on a 10 scale). This simple fact is a sign of the pressure I believe.

 

[Vierran]

I like to look at the reviews for games but they don't factor in majorly with my purchase chances especially not scores, scores are useless since most people never tell you what there scores equal, for instance is 10 a perfect game? near perfect? perfect for genre? in the end it is just as much opinion as the rest of the review so i don't put too much weight into it.

 

[LTK_70]

I frequently visit attention to Metacritic, and before buying a game I always check its score. But I could have told you myself what came out of that survey. Aggregate scores do factor in my decision whether or not to buy a game, but obviously, if a RTS game is praised into heaven by the critics, that's not going to change the fact that I don't generally enjoy RTS games, so it's not likely I'm going to buy that game. And frankly, if a game like STALKER: Call of Pripyat or Deus Ex 3 doesn't get good reviews, I'm still going to get it because I completely dig the setting, and I want to know how they continued it. Not really a big surprise here.

 

[Pimppeter2]

I think hype plays a bigger role than actual reviews

 

[hansari]

Or, as Creutz said, they should just concentrate on making a great game instead of worrying about what Jeff Gerstmann [http://en.wikipedia.org/wiki/Jeff_Gerstmann] happens to write about their game.

As much as I respect the mans opinion and enjoy his new website, his review of Kane and Lynch doesn't seem to have hurt the game enough to stop a sequel...or its initial sales... [http://vgchartz.com/games/game.php?id=7739&region=All]

 

[Dr. Gorgenflex]

Wow, I don't buy a game unless I read every review out there, I thought more people were like me. I knew alot of sites are corrupt, which is why I read nearly everyone.

 

[Nova5]

Don't really bother with reviewers who use scores. Then again, I only really read/watch reviews here on the Escapist, so big surprise.

Have to say, it's amazing it took so long for someone to say this. And if devs haven't figured this out, Christ they're not paying attention to their own customers.

 

[Laura.]

I don't care for reviews, all I do is search for gameplay videos on youtube.

 

[Thesreyn]

Correlation does not equal causation.

Surveys are amongst the worst forms of information collection methods in empirical research.

Mr Analyst "proves" nothing.

 

[Jharry5]

I usually go on recommendation from friends as opposed to game review scores. It's a 'try before you buy' kind of thing.
Genre and past games (if it is a series, that is) are also bigger factors for me instead of review scores.