The simple solution to the Metacritic problem

[tippy2k2]

In the second part of my post, I stated that even if you forced me to choose ten games (which your original post STILL says you don't have to do that, though you are saying here that you do have to do that)

No, you don't HAVE to rate 10 games, but if you do then your highest score will be higher and the lowest will be lower (the bold part being of interest to bombers).

I hit TAB on the top ten items and rate them whatever. I suppose if you really need me to test this I can but I'm thinking a minute at most.

So let's say we've added a minute and thirty seconds (plus the time it takes to make a new email account) to the time it takes to review-bomb a game with a single "zero" score. I'd say that'll reduce the number of bombs. Sure, some will still do it, but less.

And yes, learning a new system is a hassle. You ever watch the hissy fits people throw when Facebook adjusts anything? This is adding another step in the process that does not hurt bombers, so yes, adding another step is a hassle to your legit users.

What extra step? For the legit users, I mean? I don't see an extra step.

Your entire new process is one big extra step.

Current way of doing it:
I find my game of choice
I click 1-10

Your way of doing it:
I find my game of choice
I click 1-10
If I now have a second game, I have to slide that game wherever, repeat until I have ten

Notice that it is one extra step compared to what I have to do now.

According to your system, if I legitimately think that MW3 is the worst game ever and deserves a 0, I have to choose ten more games to put on my list in order to reflect this. If I'm a bomber, I don't care and will just fuck up ten random scores to smack MW3 down. If I'm a legit user, I now have nine more games that I'm going to have to review in order for my true opinion to be heard. Now a legit user has nine extra steps to take.

The mistake you're making here is assuming that adding two minutes of a troll's review is going to stop them from bombing. It won't. All you are doing is smacking legit users.

... What?

1) Open Account
2) Write a review for a game.
3) Write a review for another game.
4) Say which one is better.

I'm really not sure what your problem is... the idea is to simply remove the idea of a score and just put the games in order from bad to good. No extra steps, and only gets complicated if you have maybe 50+ games, and you can't decide where the new one should go.

If you are a bomber, you'd have to input a lot of games to make a difference. Make a new account with just one game, and that game will be considered to be the best by the system.

You have Metacritics biggest problem; too focused on scores.

I think it's a good idea. Sure it may be dificult to make comparisons between RTSs and FPSs, but surely that would just show what the user likes. That's good data to have... you could finally show which game genre is considered the best, without the use of market data.

I don't know what my problem is either because I guess I don't understand his system. All I see is a system that makes a legit user do more work to put their reviews in but not enough work to make a bomber turn back.

It might just be my misunderstanding of his idea so I'll just sit here in the corner and watch.

 

[Epic Fail 1977]

Now, if you asked me to "officially" say which one is better, I couldn't do it because it depends on whether you're looking for an RPG or a racing game.

This is what I was talking about in the post above. I always thought that your user score was for you to say which one you think is better. Sorry to break it to you, but you're not an "official" reviewer (and even if you are, your review should then be in the critics section, not the users section).

Oh....user score.....well, don't I feel like a jackass now.

Nevermind, I thought you were talking about professional reviewer scores.

So yeah, this is a decent idea then. (retreats back to the corner in shame)

Crap, another edit to the OP. I wonder how many other people I confused :S

 

[Dutchy115]

There isn't any easy total solution to the problem, but there are a few minor adjustments that could be made to make the system fairer.
For example, for games that are widely known to have nemesis titles both with unhealthy, endlessly-dedicated extremist fan-bases (such as MW3 + BF3), users will not be permitted to give one game a 10 and the other a 0, as the only real explanation for such action is an extremely biased p.o.v on the behalf of the reviewer, the fact is, if you check the accounts of user?s reviewing on both game?s pages, they tend to have all given their preferred game a 10 and its competitor a 0. Alternatively, users could be given the option of flagging reviews for deliberate score inflation or deflation (i.e. A ?0? review for MW3 with the text ?AGDSGFDHSFGDSHDAGSHSAGFDSFGDSAHFD? or a ?O? review for MW2 with the text ?I?m appalled this game has such a high user-score, hopefully this will bring it down a bit?) this would also work for games given a ?10? without sufficient justification.
My Metacritic username is DarkEnergon, I like to think I review all games/movies fairly, with two exceptions, my reviews for Transformers: Dark of the Moon and MW3, which I gave a 9 and a 10 respectively, which upon current analysis, were somewhat biased or at the very least blinded by all the clap-trap reviews hitting the pages, I intend to change these scores a.s.a.p.

 

[Jaime_Wolf]

Though it doesn't solve the biggest problems, this is still a really nice idea that lends itself well to exactly the sort of thing most people use reviews for - trying to decide whether a game is worth buying or trying to decide which of a small list of games is most worth buying.

Reviews are already treated as ordinal data, despite not being gathered in that way, and this would ensure that the data actually IS ordinal. It makes a lot of potentially useful data analyses much easier too.

This would not, however, solve the larger problems with Metacritic. People could still put games on the bottom of the list simply out of spite/fanhate/etc and, if scores were actually weighted based on the number of titles in a list (being on the bottom of a thousand is worse than being on the bottom of ten), people could use that to unfairly influence the system.

What Metacritic needs to do is normalize scores across reviewers/accounts. So if you consistently try to manipulate the rating by scoring things exceptionally low or high, it won't matter at all.

 

[Talux]

Why not stop worrying about scores or ranks and actually, you know, read or watch a review?

CoD 7 or BF3 getting a nine or a ten is completely meaningless to me. It's just an arbitrary number. The same goes for all the ridiculous review bombing. A lot of the time I just wish people would cut out review scores entirely or at least bring them down to fairly simple star ratings. Saying that CoD7 is a '9' is completely meaningless to me compared to someone describing how tight the controls are, how well balanced the weapons and maps are, whether or not it's innovative at all, whether the single player campaign is worth the effort and so on.

 

[lastoftheline]

I think the issue is less with how metacritic works (i think it's beautiful) and morw with a strange view that gamers have, especially thoe who queued up at midnight on the night of release of MW3.

It seems they expect that the game almost belongs to them, and have thus gotten up-set that Infinity ward, and moreover activision have deemed to release pretty much the exact same game that they have the previous two times. Is it really unexpected considering the fact THEY ALL STILL BOUGHT IT!?!?!

Why fix what makes you a billion dollars?

I'm sure MW3 stands up as a game on it's own quite well, why everyone is having a fit about it don't know. Some gamers need to get over themselves, beause I could bet a large number of those who have given it a 1 are still playing it and haven't seen the sun for three weeks!

/rant

 

[Yopaz]

That doesn't make sense because the scores would only be relative to other stuff.

You could have a gem of a game that would be ranked badly relative to other good games, or an average game getting a great rank because it's in a group of games that are complete ass.

If the gem really is so great then why did you rank it so low?
And if all your games are complete ass in your opinion (which is what your metacritic score is supposed to reflect) then why are you even a gamer?

The ranking system doesn't indicate how good a game is. If I have games ranked in the following order. (Not an actual list, just a random compilation of good games to make a point)
Legend of Zelda Skyward Sword
Skyrim
Batman Arkham Asylum
Saints Row The Third
Deus Ex Human Revolution
Alone In The Dark
Duke Nukem Forever
With this list it would indicate that Alone In The Dark is almost as good as Deus Ex. I have played both games and trust me. They are not equally good. Where Deus Ex is a gem Alone In The Dark's only selling point is that you can skip through the story if you don't want to play it. This system would not work.

 

[Smooth Operator]

I have a far better one that will solve the score shenanigans once and for all, remove all scores.

 

[teebeeohh]

waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.

 

[Rednog]

The simplest solution to metacritic is to simply just get rid of it.
The system is completely flawed and with various legit reviewers not actually working on the same rating scale leads to problems and then users are well....users, and you really can't solve people's stupidity.

 

[The Woolly One]

The way to do it would be to delete user reviews that are too much higher or lower than the critic reviews. Scores that fall outside of this range would be ignored. For example, if critics rate a game as 6/10 then the user scores should be expected to fall between about 4 and 8, with a few outliers obviously, but statistically the majority ought to be around there so scores outside of that kind of range seem suspect. This would also stop people 'zero-bombing' good games out of silly reasons.

For example, people who gave Portal 2 zeroes were wrong. It wasn't their opinion, they were just wrong - zero represents a game that is entirely unplayable and cannot really be described as a game. They gave it a zero for things like day one DLC despite the fact that the game itself was amazing. Likewise with MW3 - people who gave it zeroes because it was the same as MW2 were again wrong. It is not unplayable, it is not broken. I don't think it should be rated as high as it is - I've played about 24 hours worth and its not significantly different to MW2 either graphically or gameplay wise. Spec ops are good, and survival is great fun, but it should lose marks for innovation. But it is a far better game than the users say - the gameplay is rivalled only by BF3, so how they can rate it as zero is beyond me.

Also, anyone who mentions the word 'average' in their review but gives a score other 4, 5 or 6 should have their review deleted or their score changed - you can't say a game is average and then give it an appalling or an amazing score.

 

[The Woolly One]

waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.

Wouldn't stop things like zero-bombing. Hundreds of people would just rate up the zero reviews.

 

[piesandwich]

I really like this idea. It would be really interesting to see how the results differ from normal ratings.
Everyone seems to be getting upset over the idea of comparing games from different genres, which I can understand. But if everyone was ranking games based on which ones they LIKED the best rather than some pressure to conclude the definitive ?best game ever? then you can rank them based on your opinion without any worry.
I think the score that would be produced from a system like this would be pretty fair, so long as people rank a good share of crappy games so the mediocre ones landed in the middle bracket rather than the bottom.

 

[teebeeohh]

waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.

Wouldn't stop things like zero-bombing. Hundreds of people would just rate up the zero reviews.

except nothing will ever stop zero-bombing, if you give people the option to rate stuff and they want to use it as a hate machine they will find a way

 

[Woodsey]

Alternatively, people could just read the fucking words, and understand that any sort of ranking/score/one-word summation at the end of a review is only there to give the idea of a ball-park consensus of people's opinions on a game.

The simple solution to the Metacritic problem would be to blow it up.

 

[Vanguard_Ex]

Guys, back the fuck up a bit, seriously. All our fellow community member is doing is suggesting a different system which, to be fair, is actually a really good idea. I'm not sure how well it would work in practice but then no system is without its holes. Calm down a bit.

 

[Vanguard_Ex]

This is possibly the stupidest idea I've heard since "Battleship movie". You can't compare games that are completely different and make one definitively better than the other, and things get more confusing when you've reviewed hundreds of games. What order would you put, say, Gears of War 2, Oblivion, Rock Band 2, Civilization 4, and Forza Motorsport 3. Not by your favorite, but by best, so your favorite genres and biases can't be a factor. Now imagine doing that with hundreds of games, all lined up in some definitive order of "best" to "worst".

Plus, this doesn't "solve" anything in the first place. The "problem with metacritic" is that people treat metacritic like it's fact so if one game gets a higher score than another, that makes it automatically better. Forcing reviewers to actually have to say which is "better" would make things worse.

The point is it's apples and oranges; different games are good in different ways, so you can't compare them in one long order. Also, it just wouldn't make any sense to say "This game gets a grade of 57/256 games." Can I stop ranting now? Do we understand why this idea is bad and you should feel bad?

EDIT: Whoops, I thought OP was talking about professional review scores, not user scores. Disregard all of that...

Wow man, seriously?

Fuck this place, really.

 

[veloper]

waaaaaaaaaaaay to complicated.
the best and only way to have the unwashed masses people assign ratings is the youtube option of thumbs up or down because nobody cares about one review out of hundreds, all people look at is the average score. and if having a number at the end of your review is so important to you feel free to ad one after the last sentence.

Wouldn't stop things like zero-bombing. Hundreds of people would just rate up the zero reviews.

except nothing will ever stop zero-bombing, if you give people the option to rate stuff and they want to use it as a hate machine they will find a way

Actually, making things more complicated for the voter, may just remove some of the poorly thought out user ratings that people here hate in metacritic.
Registering new alts to bomb a game only let's you put them at #1 with zero relevancy.

Displaying the current ranked list in front of the voter, may even help putting games in perspective: even if the voter is a hater, there's always something more hated.

I think the idea has alot of merit.
Rating if done consistently, is similar to ranking, only with fewer positions and possible gaps. A better game should never get a worse score and a reviewer who cannot even compare similar games has no business doing reviews anyway.
No game exists in a vacuum and even reviewing a game on it's own merits, only takes the originality factor of who did it first, out of the equation. Excellent and poor are still relative only to experiences with other games.

Ranking has alot of beneifts and no real downsides:
1. the top and bottom ranks are only extreme, for users who have already voted alot of games, so trolls registering new alts to bomb a particular game have no impact

2. it takes effort to build up a personal list form scratch

3. ranking is a transparant system and people who make random lists with a target at the end, can easily be recognized and removed

4. a good reviewer should always suggest better alternatives for a poor game and this scheme makes that an automatic

5. people who have played more games should have a better perspective on games and this scheme makes their votes more weighted.


The only thing that needs to be worked out is the categories. Comparing shooters to shooters works, but comparing them to puzzle games, not so much.

 

[maninahat]

Frankly, it is a terrible idea.

1. How easy is it to compare two seperate games in two totally different genres?
2. And what happens when a reviewer reviews hundreds of games? I can't imagine it being very simple or practical.

1. How is it not easy? Either you like A better than B, or B better than A. It's no more difficult than the current system.
2. You mean because the list would be too long to fit on one screen?

1. It ain't easy for the reason I stated: if the games tell very different stories, have totally different gameplay, from different periods in time, and are in totally different genres, then how exactly is it helpful for reviewers to throw them into some kind of order? Is Galaxian better than COD4? Is Portal better than Pac Man? Instead of just judging a game on its own merits, this system requires gamers to judge a game on another, totally different game's merits as well.
2. Are we just including 10 games, or 1000 games? If a game weighs in at number 789, could it be considered a shitty game, or is it just a very enjoyable game that happens to be competing against 788 marginally better games? You can't tell from the ranking. Basically this system encourages people to only buy stuff in the top few, and disregard anything that is too low in the list. That's going to suck for niche genres (like flight simulators) because they will inevitably get ranked lower by all the reviewers who prefer their FPSs over Sims.

The whole point of a review is to tell you if the game is good, not if it is better. If a reviewer wants to come up with a top 10 favourite games or recommend similar titles to the one being reviewed then fine, but they should only be doing it to suppliment their ordinary reviews.

 

[tehroc]

heres a thought... you could ignore user reviews. personally i dont see why they are there, they are obviously not accurate.

User scores are there cause it's entirely too easy to buy good critic ratings. 99% of every professional game reviewer will be corrupted, it's all about free games and playing them before release for those types.