The simple solution to the Metacritic problem

[Athinira]

The simple solution to the Metacritic problem: READ the reviews instead of reading the score.

A high score doesn't mean you'll like a game.
A low score doesn't mean you'll hate a game.
A terribly written user review full of hate and with a 0 out of 10 is hardly going to justify the games true quality.
A well written critic review full of praise and with tendencies to discard faults is likely rating the game higher than it deserves.
A well-written, almost completely non-biased review with many good points is not going to help you if it's a genre you don't like or you disagree with the points being made.

TL;DR: As always, the simplest solution is actually using your brain.

 

[veloper]

Uh . . . no.
So if I play a completely shit game, and only that game, then that's my #1 ranked game?

In that case you shouldn't be handing out scores at all.
If all you play is one shitty game, you're not the person to recommend games to anyone.

 

[teebeeohh]

Actually, making things more complicated for the voter, may just remove some of the poorly thought out user ratings that people here hate in metacritic.
Registering new alts to bomb a game only let's you put them at #1 with zero relevancy.

Displaying the current ranked list in front of the voter, may even help putting games in perspective: even if the voter is a hater, there's always something more hated.

I think the idea has alot of merit.
Rating if done consistently, is similar to ranking, only with fewer positions and possible gaps. A better game should never get a worse score and a reviewer who cannot even compare similar games has no business doing reviews anyway.
No game exists in a vacuum and even reviewing a game on it's own merits, only takes the originality factor of who did it first, out of the equation. Excellent and poor are still relative only to experiences with other games.

Ranking has alot of beneifts and no real downsides:
1. the top and bottom ranks are only extreme, for users who have already voted alot of games, so trolls registering new alts to bomb a particular game have no impact

2. it takes effort to build up a personal list form scratch

3. ranking is a transparant system and people who make random lists with a target at the end, can easily be recognized and removed

4. a good reviewer should always suggest better alternatives for a poor game and this scheme makes that an automatic

5. people who have played more games should have a better perspective on games and this scheme makes their votes more weighted.


The only thing that needs to be worked out is the categories. Comparing shooters to shooters works, but comparing them to puzzle games, not so much.

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.
and wouldn't this lead to massive flaming, not for my rating but because of the way i categorize the games i review?
also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

 

[veloper]

Actually, making things more complicated for the voter, may just remove some of the poorly thought out user ratings that people here hate in metacritic.
Registering new alts to bomb a game only let's you put them at #1 with zero relevancy.

Displaying the current ranked list in front of the voter, may even help putting games in perspective: even if the voter is a hater, there's always something more hated.

I think the idea has alot of merit.
Rating if done consistently, is similar to ranking, only with fewer positions and possible gaps. A better game should never get a worse score and a reviewer who cannot even compare similar games has no business doing reviews anyway.
No game exists in a vacuum and even reviewing a game on it's own merits, only takes the originality factor of who did it first, out of the equation. Excellent and poor are still relative only to experiences with other games.

Ranking has alot of beneifts and no real downsides:
1. the top and bottom ranks are only extreme, for users who have already voted alot of games, so trolls registering new alts to bomb a particular game have no impact

2. it takes effort to build up a personal list form scratch

3. ranking is a transparant system and people who make random lists with a target at the end, can easily be recognized and removed

4. a good reviewer should always suggest better alternatives for a poor game and this scheme makes that an automatic

5. people who have played more games should have a better perspective on games and this scheme makes their votes more weighted.


The only thing that needs to be worked out is the categories. Comparing shooters to shooters works, but comparing them to puzzle games, not so much.

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.

Yes it would. Maybe the cut-off point wouldn't be at 10, but it makes sense that if someone is to recommend a shooter to the public, he's played more than just one.

and wouldn't this lead to massive flaming, not for my rating but because of the way i categorize the games i review?

I don't think the users would get to categorize the games themselves.
This could lead to the situation where someone might want to expand his shooter list with Mass Effect 2, but cannot because ME2 is listed as role-playing on metacritic and the site would get some flack for it. So be it.

also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

This actually very simple: a system can normalize all the user rankings for you, above the cutoff point, onto a single percentile scale.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

And why would you have it any other way? What value is the recommendation of some user who doesn't even make an effort?

 

[teebeeohh]

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.

Yes it would. Maybe the cut-off point wouldn't be at 10, but it makes sense that if someone is to recommend a shooter to the public, he's played more than just one.

so if i write a bunch of good reviews for let's say RPGs and i buy one shooter and praise it for good gameplay, amazing story, interesting characters and motivation MP, my word has less merit than the word of the guy who wrote "OMG COD IS TEH ROZZ0Rs" as a review for every CoD game ever just because he wrote more reviews?
shouldn't i be able to recommend a game because it have played tons of games and can tell if a game is good an engaging without having played others like it?

also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

This actually very simple: a system can normalize all the user rankings for you, above the cutoff point, onto a single percentile scale.

yeah my problem with that is that right now it's clear where the score comes from (total score of all reviews divided by number or reviews) and while i get how you would get a normalized score out of the ranking system it wouldn't be as easy to get as the current system.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

And why would you have it any other way? What value is the recommendation of some user who doesn't even make an effort?

because sometimes "sucked, controls" is all i need to know. Besides, ranking seems to encourage writing a lot of reviews, not writing good ones. In order to give my reviews meaning i could just write 10 one-line reviews and be done with it and that's what people will do if one review is important to them. It also sounds incredibly elitist to say that unless people have spend X hours writing reviews they will be ignored, and i am a PC gamer.

Why not split the user reviews into a review and rating section? in the rating section the trolls and hatebombers can thumb up/down a game to get a nice big green/red bar and in the actual review section people can write proper reviews using the ranking system (which i think is a great idea, i just don't think it will work well metacritic).

 

[Jandau]

NOTE: THIS IS A SUGGESTION FOR USER SCORES ONLY, not critic scores

Hard to imagine this hasn't been said before, somewhere, but anyway...

Instead of allowing users to assign a score, only allow them to assign a rank (relative to games they've previously ranked) and derive the score from that. So if I have ranked 9 games on metacritic then the highest would be scored let's say 9, the next 8, and so on down to the lowest, which would get 1. When I decide to rank a 10th game, I can't assign a score to it, I can only say where it ranked relative to the other 9. Metacritic could then adjust the scores for all 10 of my games accordingly. It's not a perfect solution, but in many ways it'd be an improvement on what they have now. Discuss.

Edited to add:
Apparently this requires further explanation.
You don't have to choose 10 games. You can have 1, or a million. You just say what order they're in. So if you have 10 games ranked, and you buy an 11th and want to rank it, then you have to say whether the new game is 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, 9th, 10th, or 11th on your list of games.

So, if I were to review/rate 10 games I really like, I would end up lowering the score for most of them? Yeah, that makes PERFECT sense...

 

[RanD00M]

Well the best solution is to disregard Metacritic completely and read reviews at other places.

 

[valleyshrew]

This is literally the single worst idea I've ever seen from the gaming community. We generally only play good games, unlike reviewers. So you'd end up having to give 1/10 to a game that got 8/10 simply because you're not going to play 7/10 and below games as there isn't enough time to waste on them. In my last 10 games, that'd probably be portal 2 getting 1/10. Really great idea!

 

[Treblaine]

NOTE: THIS IS A SUGGESTION FOR USER SCORES ONLY, not critic scores

Hard to imagine this hasn't been said before, somewhere, but anyway...

Instead of allowing users to assign a score, only allow them to assign a rank (relative to games they've previously ranked) and derive the score from that. So if I have ranked 9 games on metacritic then the highest would be scored let's say 9, the next 8, and so on down to the lowest, which would get 1. When I decide to rank a 10th game, I can't assign a score to it, I can only say where it ranked relative to the other 9. Metacritic could then adjust the scores for all 10 of my games accordingly. It's not a perfect solution, but in many ways it'd be an improvement on what they have now. Discuss.

Edited to add:
Apparently this requires further explanation.
You don't have to choose 10 games. You can have 1, or a million. You just say what order they're in. So if you have 10 games ranked, and you buy an 11th and want to rank it, then you have to say whether the new game is 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, 9th, 10th, or 11th on your list of games.

Nice idea, but it doesn't belong on metacritic.

My suggestion: file for a patent for the system and try it on your own website. There is no reason for this to tie in with metacritic which IS an "X/10" website, it just is.

 

[Jitters Caffeine]

I would say this is FAR from a simple solution to ANY problem. But I think I know what you're going for. A problem I see though is what happens if you have a LONG list of games you score? Do your scores count LESS if your list is longer? Like if you have a list of 10 games, and you put a game in the 10th place, that means it's bad. But if you have a list of 100 games and you put a game at 10, that would make it fairly good. Wouldn't that complicate your system a little?

 

[veloper]

so i need to play 10+ shooters for my tf2 rating to actually mean something? Even if it's the only shooter i play? that would make my vote for the game almost irrelevant even if i write a good, well done review full of crunchy points that have a lot of merit.

Yes it would. Maybe the cut-off point wouldn't be at 10, but it makes sense that if someone is to recommend a shooter to the public, he's played more than just one.

so if i write a bunch of good reviews for let's say RPGs and i buy one shooter and praise it for good gameplay, amazing story, interesting characters and motivation MP, my word has less merit than the word of the guy who wrote "OMG COD IS TEH ROZZ0Rs" as a review for every CoD game ever just because he wrote more reviews?
shouldn't i be able to recommend a game because it have played tons of games and can tell if a game is good an engaging without having played others like it?

For the purpose of aggregate scores, yes, it has less merit.
The reason for this is that while your review may be an excellent piece of prose, there's no way for a system to quantify this value. This is no different under the current system. Under the current system, you could have just copy/pasted something(anything) and still be a single vote bomber or do a 10/10 for a company. Nolonger.

What does change here is that your COD fan will have to rank the COD games in order of preference, instead of giving all of them the maximum score. It's not unreasonable to suppose that the aggregate of all COD fans may actually be a fairly good judge on the relative quality between those sequels.

also: if i just punch in the name of a game what will i see and how will this help me decide if a game is good? the system seem to needlessly complicated to just give me a number and even if it did i doubt it would be possible for the average user to determine where this number comes from.

This actually very simple: a system can normalize all the user rankings for you, above the cutoff point, onto a single percentile scale.

yeah my problem with that is that right now it's clear where the score comes from (total score of all reviews divided by number or reviews) and while i get how you would get a normalized score out of the ranking system it wouldn't be as easy to get as the current system.

It could still be just as clear where the score comes from. There's various way to do weigh the scores and one simple way is to display the following formula: user_score = 100% * (sum(points)/sum(listlenght))
What happens here is points get assigned inversely to the rank, so if a user has 20 games reviewed, the #1 game gets 20 points. Now someone who has reviewed alot of games makes a bigger impact, which is what I wanted.

and wouldn't this discourage people from participating because you are basically putting up a huge sign saying: "you must devote this much time to participate", creating an oligarchy of rating that will eventually make metacritic irrelevant except for people who are too lazy to calculate an average out of all the professional review sites themselves?

And why would you have it any other way? What value is the recommendation of some user who doesn't even make an effort?

because sometimes "sucked, controls" is all i need to know. Besides, ranking seems to encourage writing a lot of reviews, not writing good ones. In order to give my reviews meaning i could just write 10 one-line reviews and be done with it and that's what people will do if one review is important to them. It also sounds incredibly elitist to say that unless people have spend X hours writing reviews they will be ignored, and i am a PC gamer.

Why not split the user reviews into a review and rating section? in the rating section the trolls and hatebombers can thumb up/down a game to get a nice big green/red bar and in the actual review section people can write proper reviews using the ranking system (which i think is a great idea, i just don't think it will work well metacritic).

I though you were arguing for long reviews instead of "sucked,controls", but it doesn't matter. The "sucked, controls" guy may be still talking out of his ass; you don't know him afteral.
All you can get out of the user scores is hoping the unreasonable extremes average eachother out and hope to get something useful out of that, which is how metacritic currently operates already. A single user review by itself means less than a bought review on a big online gaming mag.

I reckon the OP's scheme would do a better job than the what we got now, exactly because it limits the impact of single votes.
That and because it would force a gradual scale on the users, instead of promoting all 10s or zeros.

 

[Epic Fail 1977]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

Thank you for trying. It's been a while since I posted a new thread at the Escapist. Now I remember why...

 

[Vanguard_Ex]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

Thank you for trying. It's been a while since I posted a new thread at the Escapist. Now I remember why...

Because now more than ever our community is a venomous den of sarcastic putdowns and scoffs? Yeah, that's why I'm hesitant to say a fair few things.
And you're welcome mate.

 

[mjc0961]

Nope. Games of different genres should not be compared in this way. It's one of the many problems with the way people use review scores.

The simple solution to the Metacritic problem is Metacritic needs to start moderating user reviews to at least some degree.

 

[tippy2k2]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

Thank you for trying. It's been a while since I posted a new thread at the Escapist. Now I remember why...

Because now more than ever our community is a venomous den of sarcastic putdowns and scoffs? Yeah, that's why I'm hesitant to say a fair few things.
And you're welcome mate.

Oh dear God are you two serious here? You threw out an idea that most people think doesn't work and now you're going to have a little pity party over it? "People were mean to my idea, they were so venomous and sarcastic and I think one of them said a racial slur!" *NOTE: For those of you who are unaware, that "racial slur" is a Family Guy joke*

If you can't accept criticism of your idea, then you're right, you shouldn't be posting. I don't see any person attacking you, just throwing out flaws in your argument. Even though I (and evidently still) don't get why your idea works better than the current system, if it's getting shot down repeatedly, there is probably a pretty good reason. Maybe no one in this forum understands your greatness?

 

[Vanguard_Ex]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

Thank you for trying. It's been a while since I posted a new thread at the Escapist. Now I remember why...

Because now more than ever our community is a venomous den of sarcastic putdowns and scoffs? Yeah, that's why I'm hesitant to say a fair few things.
And you're welcome mate.

Oh dear God are you two serious here? You threw out an idea that most people think doesn't work and now you're going to have a little pity party over it? "People were mean to my idea, they were so venomous and sarcastic and I think one of them said a racial slur!"

If you can't accept criticism of your idea, then you're right, you shouldn't be posting. I don't see any person attacking you, just throwing out flaws in your argument. Even though I (and evidently still) don't get why your idea works better than the current system, if it's getting shot down repeatedly, there is probably a pretty good reason. Maybe no one in this forum understands your greatness?

Haha, all I can really say is thank you for proving my point. And what the fuck are you even talking about with this racial slur stuff?! There's sarcasm and there's...that. Try harder mate.

It isn't a problem with criticism that's the issue here, it's the innately hostile reaction of an idea that community members merely disagree with. A lot of the guys here now act like someone's said the most retarded thing on the planet when all they need to state is that it isn't a good idea. Not attacking the member on a personal level, you know this is true regardless of how much you're going to come back with more unnecessary, catty sarcasm. But like I said, way to prove my point by acting like the social equivalent of a plague rat.

 

[tippy2k2]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

Thank you for trying. It's been a while since I posted a new thread at the Escapist. Now I remember why...

Because now more than ever our community is a venomous den of sarcastic putdowns and scoffs? Yeah, that's why I'm hesitant to say a fair few things.
And you're welcome mate.

Oh dear God are you two serious here? You threw out an idea that most people think doesn't work and now you're going to have a little pity party over it? "People were mean to my idea, they were so venomous and sarcastic and I think one of them said a racial slur!"

If you can't accept criticism of your idea, then you're right, you shouldn't be posting. I don't see any person attacking you, just throwing out flaws in your argument. Even though I (and evidently still) don't get why your idea works better than the current system, if it's getting shot down repeatedly, there is probably a pretty good reason. Maybe no one in this forum understands your greatness?

Haha, all I can really say is thank you for proving my point. And what the fuck are you even talking about with this racial slur stuff?! There's sarcasm and there's...that. Try harder mate.

It isn't a problem with criticism that's the issue here, it's the innately hostile reaction of an idea that community members merely disagree with. A lot of the guys here now act like someone's said the most retarded thing on the planet when all they need to state is that it isn't a good idea. Not attacking the member on a personal level, you know this is true regardless of how much you're going to come back with more unnecessary, catty sarcasm. But like I said, way to prove my point by acting like the social equivalent of a plague rat.

Yes, the racial slur thing was a joke. From now on, I will make sure to mark those for your convenience.

We are attacking his idea because we don't think it's a good idea. If you don't want your idea attacked, then yes, don't post it. Then, after we do attack it, there is a "OH whoa is me! How could they not recognize the greatness of my idea!" post and I called you on it. You come back with a personal insult. Well played sir, well played...

But this thread is not about you two and your complaining that everyone is making fun of your idea so I'm going to go ahead and not respond to you anymore. Go ahead and put out your next idea and I'll make sure to do nothing but agree with you since disagreeing is venomous and mean.

 

[Vanguard_Ex]

Yes, the racial slur thing was a joke. From now on, I will make sure to mark those for your convenience.

We are attacking his idea because we don't think it's a good idea. If you don't want your idea attacked, then yes, don't post it. Then, after we do attack it, there is a "OH whoa is me! How could they not recognize the greatness of my idea!" post and I called you on it. You come back with a personal insult. Well played sir, well played...

But this thread is not about you two and your complaining that everyone is making fun of your idea so I'm going to go ahead and not respond to you anymore. Go ahead and put out your next idea and I'll make sure to do nothing but agree with you since disagreeing is venomous and mean.

I'm aware, it was just a terribly unusual joke. It didn't really seem to fit into what was being said, at all. Also you seem to think this idea was partly my doing, which, well, it wasn't...anyway.

No no, once again, attacking the idea is not the problem I have. If you came in to say the idea is bad, fine, that isn't my gripe. I'm not defending criticism of the concept. What is my problem is the number of people in this thread attacking the OP personally simply because they don't like my idea. To reiterate so you drop the painfully childish sarcasm: disagreeing with an idea, fine. Personally attacking the OP or otherwise acting like a complete **** to him because you don't like the idea, major problem I have with the community nowadays.

 

[The3rdEye]

NOTE: THIS IS A SUGGESTION FOR USER SCORES ONLY, not critic scores

Hard to imagine this hasn't been said before, somewhere, but anyway...

Instead of allowing users to assign a score, only allow them to assign a rank (relative to games they've previously ranked) and derive the score from that. So if I have ranked 9 games on metacritic then the highest would be scored let's say 9, the next 8, and so on down to the lowest, which would get 1. When I decide to rank a 10th game, I can't assign a score to it, I can only say where it ranked relative to the other 9. Metacritic could then adjust the scores for all 10 of my games accordingly. It's not a perfect solution, but in many ways it'd be an improvement on what they have now. Discuss.

Edited to add:
Apparently this requires further explanation.
You don't have to choose 10 games. You can have 1, or a million. You just say what order they're in. So if you have 10 games ranked, and you buy an 11th and want to rank it, then you have to say whether the new game is 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, 9th, 10th, or 11th on your list of games.

IMHO, Systems have two functions: To make things easier for some people, and to give others something to break. It would be very easy given your system for me to "tag" games like "Barbie Horse Adventures", "Spongebob goes to the Beach", "Desert Bus", etc and then when a title came out that I wanted to see destroyed I could "tag" it and put it at the bottom of the list. I'd hazard the guess that AAA games will always be a minority in terms of the number released compared to the market as a whole so there will always be plenty of ammunition for people to shoot them down. It's the John Gabriel Greater Internet Fuckwad theory all over.

Normal Person + Audience + Anonymity = Total Fuckwad

You can't test to see if someone's normal because that would be discriminatory.
You can't remove the audience because that's who you're trying to inform.
You can't remove the anonymity... actually you can but you should first ask Blizzard how the entire ReadID thing worked out.

People are dicks basically. If someone wants to find a source of reviews that they will find reliable, they need to find a site who's -overall- format matches their own approach and expectations, not a bunch of random Romans with one thumb pointed down and the other up-

 

[tippy2k2]

Yes, the racial slur thing was a joke. From now on, I will make sure to mark those for your convenience.

We are attacking his idea because we don't think it's a good idea. If you don't want your idea attacked, then yes, don't post it. Then, after we do attack it, there is a "OH whoa is me! How could they not recognize the greatness of my idea!" post and I called you on it. You come back with a personal insult. Well played sir, well played...

But this thread is not about you two and your complaining that everyone is making fun of your idea so I'm going to go ahead and not respond to you anymore. Go ahead and put out your next idea and I'll make sure to do nothing but agree with you since disagreeing is venomous and mean.

I'm aware, it was just a terribly unusual joke. It didn't really seem to fit into what was being said, at all. Also you seem to think this idea was partly my doing, which, well, it wasn't...anyway.

No no, once again, attacking the idea is not the problem I have. If you came in to say the idea is bad, fine, that isn't my gripe. I'm not defending criticism of the concept. What is my problem is the number of people in this thread attacking the OP personally simply because they don't like my idea. To reiterate so you drop the painfully childish sarcasm: disagreeing with an idea, fine. Personally attacking the OP or otherwise acting like a complete **** to him because you don't like the idea, major problem I have with the community nowadays.

Alright, I said I wasn't going to but I'm going to respond one more time. I 100% agree that IF people were attacking the OP or acting like a d-bag to him would not be acceptable. I went with my "Pity Party" shot because it looks like you were complaining about people attacking his idea. Looking over the thread, I do not see the personal attacks (there are a lot of pages however so maybe there is some and I'm not seeing it). If that is what you were complaining about, then I do apologize, it just sounded like you were complaining about his idea being ripped apart.

EDIT: Wow, am I misinterpreting in this thread. First his idea (though I still think it's a bad idea once someone cleared it up for me) and you now.