Metacritic Adds, Then Pulls Developer Review Scores

[Logan Westbrook]

Metacritic Adds, Then Pulls Developer Review Scores

The site was trying to be helpful to its users, but faulty information hampered its efforts.

This week, review aggregator Metacritic started posting scores for developers, based on the average metascore of the games they'd worked on. The scores were part of Metacritic's relaunch last August, but were recently pulled shortly after gaining greater public awareness. This left more than one person confused, prompting an explanation from the site's games editor, Marc Doyle.

Writing on the Metacritic blog [http://features.metacritic.com/features/2011/site-news-march-28-2011/], Doyle said that the reason the scores went up in the first place was to help people find new games that they might enjoy. The plan was that users would be able to click on the creator of a game they liked, and see what else he or she had worked on. Doyle noted that it already worked that way for the Metacritic's movie, TV, and music reviews, and that the site wanted to add the same functionality to the games section.

Unfortunately, it transpired that the information that the site had for videogames wasn't as accurate as the details it had for other media. For movie credits, Doyle wrote, Metacritic licensed the information from IMDB, but for videogames, it came from Metacritic's sister site, GameFAQs, which contained a much higher rate of inaccuracies.

Doyle said that Metacritic was committed to creating a credits database for videogames, but acknowledged that the information it was using was incomplete at best. The scores will be making a return - mores the pity - but not until Metacritic is convinced that the information is correct.

Source: via CVG [http://www.computerandvideogames.com/295800/news/metacritic-pulls-developer-ratings/?cid=OTC-RSS&attr=CVG-News-RSS]




Permalink

 

[Natdaprat]

I suppose it would take a few dedicated and informed editors to keep this up to standards. Overall a good feature that will hopefully keep the general populace aware of a developers catalogue, and know that it's not the same company that makes Call of Duty games annually.

 

[DTWolfwood]

sounds like a good idea. GL with that endeavor. Might give me more reason to visit the site than the occasional lolz i get from looking at the critic/user score, DA2 for example XD

 

[tkioz]

Sounds like a good idea if they get it right. I already do that on Steam, if I find a game I like, I generally look at the development team that made it and see what other games they've got up on Steam.

 

[LavaLampBamboo]

This is the sort of thing I could really get behind. I hope they look into this more, and put some funding behind it to get it established.

Is there perhaps a need for an IGDB?

 

[ZeppMan217]

This is the sort of thing I could really get behind. I hope they look into this more, and put some funding behind it to get it established.

Is there perhaps a need for an IGDB?

I do believe guiz at IMDB are already working on something like that.

 

[ShadowKirby]

Great, they're going to make averages of already inflated scores.

 

[Panayjon]

lol GameFAQs.

Its a useful site but the forums there make 4Chan look like an intelligent and wonderful place.

 

[MrJoyless]

Inaccuracies on metacritics website NEVER!!!

Companies and devs bounce around / get remade so much now its hard to judge a companies future success with past products anymore to any large extent

 

[Taerdin]

One I noticed was that they had SiN Episodes: Emergence down as a Valve game (real dev: Ritual Entertainment) probably because it's on Steam and the only person who could be bothered to put the developer thought that every game on Steam is developed by Valve or something.

 

[UNHchabo]

One I noticed was that they had SiN Episodes: Emergence down as a Valve game (real dev: Ritual Entertainment) probably because it's on Steam and the only person who could be bothered to put the developer thought that every game on Steam is developed by Valve or something.

The game also used the Source engine.

 

[Taerdin]

Actually if you look at this list : http://www.metacritic.com/company/valve-software you will notice that Valve has also got Sam & Max Season One under their belt. Man Valve is awesome.

 

[Sparrow]

That actually sounds like a great idea. I'm guessing they either already do or are planning to do this for directors and film companies?

 

[Ruptuk]

Had a look at this weeks ago. The infomation is pretty awful tbh, my entry doesn't accurately reflect my history but its worse on IMDB so thats a plus .

 

[Jamie Doerschuck]

Yeah, but RANKING the developers and listing all the games they've worked on are two different things..

I also feel that the Doyle guy is being a little mis-leading. If you read what he said, he never really mentioned the fact that the individuals were ranked. He just makes it out to be, again, that if you click on their name you get a list of their other games. Not that they've taken the average of all those games and used it to assign a number to an individual..

I don't really think that's a fair thing to do. Even if we did pretend that Metacritic scores were always accurate, these people rarely ever work with the same people consistently. Maybe one game was just bad because there was miscommunication between the head and the rest of the team. No one really knows what happens to an individual game in the years it's being developed (unless you worked on it, of course).

 

[D_987]

sounds like a good idea. GL with that endeavor. Might give me more reason to visit the site than the occasional lolz i get from looking at the critic/user score, DA2 for example XD

Sounds like a good idea if they get it right. I already do that on Steam, if I find a game I like, I generally look at the development team that made it and see what other games they've got up on Steam.

This is the sort of thing I could really get behind. I hope they look into this more, and put some funding behind it to get it established.

Is there perhaps a need for an IGDB?

I think you people either need to look up what they were really doing, or just why it's such a bad idea...of course this statement, and consequently the article, is full of PR spin - [though the "Mores the pity" line ensued The Escapist clearly think it's a bad idea too] - but they were actually assigning individual people a "score" based on all the games they had worked on regardless of position. Of course this score being from Metacritic it was an average.

There're a million and one problems with this system, the primary worry being that publishers would start using it to hire people with; they already attach bonuses and targets to studios to gain specific Metacritic scores, and base what games they'll find around these scores - it seem the logical step they'd take. Furthermore the fact it's an average meant those that had worked on quality titles with a few poor ones they weren't actually involved in directly [say the "Producer" of a port] - their score would drastically drop; Miyamoto's score was an 80, for example, despite the games he had worked on as designer being much higher than those he produced.

The way Metacritic assigned the scores to position ratio meant that the position was very unclear - it was clearly all about the score - and Metacritic at the moment when scoring games is terrible [see the 100 they gave The Escapists review of Dragon Age, despite the 5 point system not representing that kind of standard at all].

If you'd like more detailed analysis just check out:

Rock Paper Shotgun [http://www.rockpapershotgun.com/2011/03/28/not-a-man-a-number-metacritic-rates-devs/]
Kotaku [http://www.kotaku.com.au/2011/03/metacritic-now-rating-humans/]

And so on - it was a terrible idea anyway, but the way they implemented it would benefit absolutely nobody - especially not the consumer; teams make games - not individuals.

 

[beema]

Oops I hit "Publish!" I meant to hit "Mark for Review!"

 

[LavaLampBamboo]

This is the sort of thing I could really get behind. I hope they look into this more, and put some funding behind it to get it established.

Is there perhaps a need for an IGDB?

I think you people either need to look up what they were really doing, or just why it's such a bad idea...of course this statement, and consequently the article, is full of PR spin - [though the "Mores the pity" line ensued The Escapist clearly think it's a bad idea too] - but they were actually assigning individual people a "score" based on all the games they had worked on regardless of position. Of course this score being from Metacritic it was an average.

There're a million and one problems with this system, the primary worry being that publishers would start using it to hire people with; they already attach bonuses and targets to studios to gain specific Metacritic scores, and base what games they'll find around these scores - it seem the logical step they'd take. Furthermore the fact it's an average meant those that had worked on quality titles with a few poor ones they weren't actually involved in directly [say the "Producer" of a port] - their score would drastically drop; Miyamoto's score was an 80, for example, despite the games he had worked on as designer being much higher than those he produced.

The way Metacritic assigned the scores to position ratio meant that the position was very unclear - it was clearly all about the score - and Metacritic at the moment when scoring games is terrible [see the 100 they gave The Escapists review of Dragon Age, despite the 5 point system not representing that kind of standard at all].

If you'd like more detailed analysis just check out:

Rock Paper Shotgun [http://www.rockpapershotgun.com/2011/03/28/not-a-man-a-number-metacritic-rates-devs/]
Kotaku [http://www.kotaku.com.au/2011/03/metacritic-now-rating-humans/]

And so on - it was a terrible idea anyway, but the way they implemented it would benefit absolutely nobody - especially not the consumer; teams make games - not individuals.[/quote]

Okay so on the face of it yeah it sounds like a good idea, and if you analyse it, then agin yes, you can see the flaws of the system. But I think if they could hammer out the kinks in it and try and solve some of the problems.

Perhaps they could show scores as how they are involved in the games, like list the average scores that Miyamoto has when he is credited as a designer alongside the scores he has when he is credited as a producer.

I realise it would affect hiring, but it think it would also help in bringing back that element of celebrity about game designers that we really don't see anymore.

 

[Twilight_guy]

*shakes fist at heaven* DAMN YOU GAMEFAQS, DAMN YOU TO HELL!

It's nice to hear that they are concerned about the quality of there data and I hope they find something slightly better then GameFAQS to get their information from. This could be quiet usefully for finding games if it works properly.