The simple solution to the Metacritic problem

[SonicKoala]

It sounds as though the system you're proposing could only be effective provided the people using it have played a wide array of games; how else could you get an accurate indication of a game's quality unless it is compared to as many other games as possible?

Take, for example, a handful of 10 people who have only played 3 games; let's say Modern Warfare 3 is one of them, and these 10 people all put MW3 in their top spot. This is going to translate as 'Modern Warfare 3 is the favourite game of 10 people'. How is this indicative of a game's quality? It just means that they liked MW3 more than those 2 other games they've played, but this will show up on the website's ranking system as 'Modern Warfare 3 is totally awesome', when this isn't necessarily what the original information was suggesting.

Furthermore, this would detract from Metacritic's use as an aggregate review website; If I go on Metacritic and see a game received a score of '90', that is a piece of information which I can readily use and make sense of. With your method, the only thing the website would communicate to me is "This game is better than these games". What if I hadn't played those other games? That piece of information would be utterly useless to me.

So yeah, I don't like your idea. Metacritic isn't perfect, but there's no sense in making it even worse by introducing some convoluted and ineffective method of assessing a game or any other product's quality.

 

[Epic Fail 1977]

Take, for example, a handful of 10 people who have only played 3 games; let's say Modern Warfare 3 is one of them, and these 10 people all put MW3 in their top spot. This is going to translate as 'Modern Warfare 3 is the favourite game of 10 people'.

And this is worse, in your opinion, than the same 10 people logging in and all rating MW3 with a 10 or a 9 on the current system? How exactly? What has been lost?

 

[SL33TBL1ND]

I'm all for written reviews more than scores, but how many times do you see coherent thought in a metacritic user review?

Not that often, huh.

So let me get this right... you want the users thoughts on their games (your words) but you don't want to read their reviews because the reviews are incoherent...

No. Read.

I'm all for written reviews more than scores

This means I like written reviews, they're more comprehensive than scores. They give us actual details.

but how many times do you see coherent thought in a metacritic user review?

This is me expressing how the average metacritic user review is horribly written and thought out.

What I want is well written thoughts and reviews. In other words, the type of things that aren't on metacritic at all because of the user-base itself.

 

[maninahat]

Frankly, it is a terrible idea. How easy is it to compare two seperate games in two totally different genres? And what happens when a reviewer reviews hundreds of games? I can't imagine it being very simple or practical.

 

[Epic Fail 1977]

I'm all for written reviews more than scores, but how many times do you see coherent thought in a metacritic user review?

Not that often, huh.

So let me get this right... you want the users thoughts on their games (your words) but you don't want to read their reviews because the reviews are incoherent...

No. Read.

I did. That's what you wrote. Shall I use the quote button, or would you like to try again? Remember, the topic is the scoring system.

 

[SL33TBL1ND]

I'm all for written reviews more than scores, but how many times do you see coherent thought in a metacritic user review?

Not that often, huh.

So let me get this right... you want the users thoughts on their games (your words) but you don't want to read their reviews because the reviews are incoherent...

No. Read.

I did. That's what you wrote. Shall I use the quote button, or would you like to try again? Remember, the topic is the scoring system.

You obviously didn't, because if you'd read the rest of that post, you'd see my clarification. But fuck it, I'm not going to debate with someone being disingenuous.

 

[Epic Fail 1977]

Frankly, it is a terrible idea.

1. How easy is it to compare two seperate games in two totally different genres?
2. And what happens when a reviewer reviews hundreds of games? I can't imagine it being very simple or practical.

1. How is it not easy? Either you like A better than B, or B better than A. It's no more difficult than the current system.
2. You mean because the list would be too long to fit on one screen?

 

[Conza]

Hard to imagine this hasn't been said before, somewhere, but anyway...

Instead of allowing users to assign a score, only allow them to assign a rank (relative to games they've previously ranked) and derive the score from that. So if I have ranked 9 games on metacritic then the highest would be scored 9, the next 8, and so on down to the lowest, which would get 1. When I decide to rank a 10th game, I can't assign a score to it, I can only say where it ranked relative to the other 9. Metacritic could then adjust the scores for all 10 of my games accordingly. It's not a perfect solution, but in many ways it'd be an improvement on what they have now. Discuss.

Edited to add:
Apparently this requires further explanation.
You don't have to choose 10 games. You can have 1, or a million. You just say what order they're in. So if you have 10 games ranked, and you buy an 11th and want to rank it, then you have to say whether the new game is 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, 9th, 10th, or 11th on your list of games.

Interesting solution. I don't really go to Metacritic that often, but I would just say force a login and registration process 1 account per email, and only 1 vote per account. Sure, they'll still be people who register 10 emails, make 10 accounts and thus have 10 votes, but I think the effort for that would deter many people.

Also, in this method of yours, no two games would be exactly equal rank. Also, lets say I only voted for 2 games, and I think both games deserve a 8/10, when I rank one first and the other second, what scores do they get?

 

[SonicKoala]

Take, for example, a handful of 10 people who have only played 3 games; let's say Modern Warfare 3 is one of them, and these 10 people all put MW3 in their top spot. This is going to translate as 'Modern Warfare 3 is the favourite game of 10 people'.

And this is worse, in your opinion, than the same 10 people logging in and all rating MW3 with a 10 or a 9 on the current system? How exactly? What has been lost?

Would your method only apply to user reviews, or would critics have to adhere to this same format? If this only applies to user reviews, than I would be indifferent, since I pay little to no attention to the user reviews.

 

[tippy2k2]

snip

snip

snip

snip

snip

snip

Yes you do. Using my method, you do, because if there's just one game on your list then it isn't ranked at all because there's nothing else to rank it in comparison to. If there's just two games then they get ranked, but it wouldn't be 0 for the lower game and 10 for the higher one. The folks at Metacritic are no strangers to math, and I'm sure they could work out something suitable, but for the sake of an example we could say that if you have just two games then the better one gets 6 and the other gets 4. To push that 4 down to a 1.1 or a 0.7 or whatever, you'd have to add more and more games to your list.

In the second part of my post, I stated that even if you forced me to choose ten games (which your original post STILL says you don't have to do that, though you are saying here that you do have to do that), that does not take long. I hit TAB on the top ten items and rate them whatever. I suppose if you really need me to test this I can but I'm thinking a minute at most.

And yes, learning a new system is a hassle. You ever watch the hissy fits people throw when Facebook adjusts anything? This is adding another step in the process that does not hurt bombers, so yes, adding another step is a hassle to your legit users.

 

[Epic Fail 1977]

Also, in this method of yours, no two games would be exactly equal rank. Also, lets say I only voted for 2 games, and I think both games deserve a 8/10, when I rank one first and the other second, what scores do they get?

I guess the system would reward players who rank lots of games by more accurately representing their opinions? But yes, it's not perfect.

 

[Epic Fail 1977]

In the second part of my post, I stated that even if you forced me to choose ten games (which your original post STILL says you don't have to do that, though you are saying here that you do have to do that)

No, you don't HAVE to rate 10 games, but if you do then your highest score will be higher and the lowest will be lower (the bold part being of interest to bombers).

I hit TAB on the top ten items and rate them whatever. I suppose if you really need me to test this I can but I'm thinking a minute at most.

So let's say we've added a minute and thirty seconds (plus the time it takes to make a new email account) to the time it takes to review-bomb a game with a single "zero" score. I'd say that'll reduce the number of bombs. Sure, some will still do it, but less.

And yes, learning a new system is a hassle. You ever watch the hissy fits people throw when Facebook adjusts anything? This is adding another step in the process that does not hurt bombers, so yes, adding another step is a hassle to your legit users.

What extra step? For the legit users, I mean? I don't see an extra step.

 

[Epic Fail 1977]

Would your method only apply to user reviews, or would critics have to adhere to this same format? If this only applies to user reviews, than I would be indifferent, since I pay little to no attention to the user reviews.

I'm only suggesting this be used for user reviews.

 

[tippy2k2]

In the second part of my post, I stated that even if you forced me to choose ten games (which your original post STILL says you don't have to do that, though you are saying here that you do have to do that)

No, you don't HAVE to rate 10 games, but if you do then your highest score will be higher and the lowest will be lower (the bold part being of interest to bombers).

I hit TAB on the top ten items and rate them whatever. I suppose if you really need me to test this I can but I'm thinking a minute at most.

So let's say we've added a minute and thirty seconds (plus the time it takes to make a new email account) to the time it takes to review-bomb a game with a single "zero" score. I'd say that'll reduce the number of bombs. Sure, some will still do it, but less.

And yes, learning a new system is a hassle. You ever watch the hissy fits people throw when Facebook adjusts anything? This is adding another step in the process that does not hurt bombers, so yes, adding another step is a hassle to your legit users.

What extra step? For the legit users, I mean? I don't see an extra step.

Your entire new process is one big extra step.

Current way of doing it:
I find my game of choice
I click 1-10

Your way of doing it:
I find my game of choice
I click 1-10
If I now have a second game, I have to slide that game wherever, repeat until I have ten

Notice that it is one extra step compared to what I have to do now.

According to your system, if I legitimately think that MW3 is the worst game ever and deserves a 0, I have to choose ten more games to put on my list in order to reflect this. If I'm a bomber, I don't care and will just fuck up ten random scores to smack MW3 down. If I'm a legit user, I now have nine more games that I'm going to have to review in order for my true opinion to be heard. Now a legit user has nine extra steps to take.

The mistake you're making here is assuming that adding two minutes of a troll's review is going to stop them from bombing. It won't. All you are doing is smacking legit users.

 

[dessertmonkeyjk]

With the whole rank system... will this rating going to constantly change depending on where they are on the list at any time? That would mean what you once thought was an excellent game goes from an 3rd to 12th position. That is a problem.

I got a better idea. Have numbered rating for each part of the game. For example, the categories are Graphics, Sound, Story, Gameplay, Aesthetics, & Misc. each with a 1-5 rating each. Once you set your ratings on these categories then they'll be average into the final score.

Simply put, you're rating the game by it's components rather then a straight forward 1-10 score. This encourages the user to think about said components but they still have the right to say which one stands out above the rest.

 

[Epic Fail 1977]

Your way of doing it:
I find my game of choice
I click 1-10
If I now have a second game, I have to slide that game wherever, repeat until I have ten

No, that's not it. Other people seem to be getting it. I'm sorry, I've tried explaining it to you a few different ways, but I think I'll stop now.

 

[RedDeadFred]

Or you could just ignore the user reviews because I'm sure 70% of them have never played the game they are reviewing.

 

[Itsthefuzz]

I'm all for written reviews more than scores, but how many times do you see coherent thought in a metacritic user review?

Not that often, huh.

So let me get this right... you want the users thoughts on their games (your words) but you don't want to read their reviews because the reviews are incoherent...

No. Read.

I did. That's what you wrote. Shall I use the quote button, or would you like to try again? Remember, the topic is the scoring system.

You obviously didn't, because if you'd read the rest of that post, you'd see my clarification. But fuck it, I'm not going to debate with someone being disingenuous.

You're right, I stopped reading at the exact point that you started trolling.

Edit: grr, can't get used to the way this forum handles quotes and edits...

I don't see any trolling, I think you're just mad at being wrong. Seemed pretty simple what he said... "Written reviews = good! Poor written reviews like you'd probably find with a user base like metacritic = bad."

 

[lord.jeff]

Not a bad idea, at the very least I'd like to see it tried out, but my one question, can your system display an average of all the ranks?

 

[Something Amyss]

heres a thought... you could ignore user reviews. personally i dont see why they are there, they are obviously not accurate.

And this, my friends, is the real simple solution.