Jimquisition: Metacritic Isn't the Problem

Dastardly

Imaginary Friend
Apr 19, 2010
2,420
0
0
Jim Sterling said:
Metacritic Isn't the Problem

Oh you poor, sad little cretins. You are all so wrong. Always. Forever. Jim Sterling illuminates your path, but what good does that do when you refuse to open your eyes? Oh, he's so much better than you.

Watch Video
By and large, I agree with what you're saying. I'd even go so far as to say your knife simile is inaccurate. This isn't like blaming knife makers for stabbings. This is like blaming a hammer maker when someone tries to use the hammer as a shovel.

But I also think that MetaCritic, recognizing the potential hazard of the tool they've created, could choose to do things a bit differently to mitigate those ill effects. Even gun manufacturers put "safeties" on their products, don't they? Taking responsibility for a solution isn't the same as taking blame for the problem.

You rightly noted that MetaCritic could use some more transparency. Letting people see a bit further into the process might allow them to make a more informed interpretation of the result. Otherwise, it's giving people an answer without really telling them what the question was. They'll create their own questions, and apply them in whatever way best suits their desires at the moment.

Of course, this also requires scoring reviewers to be more transparent about what their numerical or letter scores actually mean. For one reviewer, a 7/10 might be a near-failing score (like a D in school). For another, a 7/10 might be phenomenal, as a 10 represents some perfect super-game that hasn't been created yet, and 70% of that ain't bad.

For many reviewers, the difference between each "step" on the scale decreases as you move up it. This is especially true of scales of 10 points or larger. Others, usually those that employ "five stars" or so, try to make each step a bit closer to symmetrical. That makes comparing and "normalizing" different scores a lot harder. I'm sure some of the folks behind a MetaCritic score are trying to take this into account, but a bit more openness on both sides would help us as consumers decide a bit better.

Of course, ultimately, the problem does rest with publishers themselves. The reason we weigh game reviews so differently from movies and music is simple: games are far more expensive. I can see a movie for $9, in its entirety. From that, I can also decide if it's worth the $20 to get it on DVD, or maybe $30 for some added features. That means, for any given movie, I can try the complete product and then purchase it, if I so choose, for half the price of a video game.

With a game, "trying" means "buying." More than with movies, we rely on others to "try" the game for us. And for the prices being asked, anything less than a glowing review is going to be seen as a "wait and buy it used" recommendation (and by the time that's feasible, the game has usually been forgotten in the hubbub of some other shiny new release). Until publishers can be swayed from the $60 price tower, that's not going to change.

Consumers are wholly justified in the weight they put on reviews. Publishers are justified in the weight they give to reviews as well, but they are not justified in their response to that assessment.

Reviewers (and MetaCritic) are intended to serve as a go-between. They are meant to communicate the merits (or faults) of a product to the consumer so that we can make good decisions, and so that both reviewers and consumers can communicate a message about our expectations to the publishers.

Consumers can only communicate via numbers (dollars and copies), which is about as fine-tunable as a game of "Marco Polo." Reviewers can communicate via narrative... but instead, we're allowing that narrative to be boiled down into just numbers, with no real context.

When communication breaks down, it doesn't matter whose fault it is. The sender has the responsibility to adjust the message until it is received clearly. MetaCritic might be distilling the message down a bit too much, to the point that the message is being sent louder, but not any clearer.

As a teacher, I know full well that sometimes students don't learn a concept because they're lazy or not trying hard enough, and sometimes it's because I didn't express it as clearly as I should. But in both cases, the responsibility is on me to be the one that says it differently or changes the stimulus until I get the result I want. So, yeah, the publishers are using it in the wrong way. They're reading the message the wrong way. Their receiving feedback in the wrong way. But they're not going to change first.

TL;DR: MetaCritic is being misused, and it's not their fault. However, knowing that they're being misused gives them a certain amount of responsibility to help change that. So while it's unwise to throw the baby out with the bathwater, it's equally unwise to just leave the baby in the bathwater. It's not unreasonable to ask MetaCritic to change how they do things.
 

a_swe_mayt_hink

New member
May 13, 2011
11
0
0
i've never used metacritic...im guessing its somewhat like imdb for games/other media sources? because you'll find that most people will make their own mind up about these sort of things. Reviewers have very little power over consumer consumption...

p.s. i think if i have to watch another stupid transformers 3 advert on this website, i'm going to have to implode.
 

RedEyesBlackGamer

The Killjoy Detective returns!
Jan 23, 2011
4,701
0
0
I agree. I love Metacritic for what it is: a place that pools reviews together so consumers don't have to search for them. The Metacritic "score" is meaningless for the most part. I'm also loving the flame bait.
 

Hybridwolf

New member
Aug 14, 2009
701
0
0
One of the more intresting shows on the escapist now but it still isn't that funny. I'd rather Jim went for breaking down the problems in gaming we face, or stopped. As although the humor is clever, it feels incredibly forced at the best at times, that violent video games joke for one. Have to admit though it has come a long way from constant MSpaint doddles and very poor material. End of the day, good show, much better then it has been and bravo for using Father Ted.

And Escapist, I appricate that you need money to survive, but I'm getting sick of this adverts which pop up and show EA games I'd have long since played if I'd wanted to. Espically the Dragon Age 2 one. I don't care about the adverts before videos, but these adverts are more then frustrating.
 

constantcompile

New member
Sep 9, 2010
61
0
0
This was painful to watch.

Even when Jim brings up a valid counterpoint, which is that focus on aggregate scoring is more a side-effect of the real problem - a broken scoring system, and those who focus on it - than the actual cause of so many other problems in the gaming industry, he's simply painful to listen to.
"You've just been served in Jim's kitchen, prick!"
When is this an appropriate response to anyone? I actually felt sympathetic to the author of the email he received - when I paused and read it, it seemed quite professional and well-written (inasmuch as an email asking someone to withhold their opinions can be). Having heard both opinions I'd rather see a video series from the author on The Escapist, instead of Jim.

Assuming Jim is a reviewer, one who sees his stances as "divisive," he himself is contributing to the problem - a good review (not a critique, a review) should take stock of the good and the bad, and if a final number score is to be given (because many people prefer a quick reference of quality), it should give a number that everyone can agree is fair; divisive or controversial reviews, almost by nature, arrive at a number that is far more distanced from popular opinion.

If an assessment of a game is overly divisive or critical, it is better termed a critique - not meant to give the reader any idea of the media's contents or quality, only the critic's opinion of them. Critics, or reviewers who consider themselves very critical, choosing to abstain from giving a number score is a good remedy for the current problem - the problem being that so much attention is paid to number scores in the first place, particularly by the gaming industry.

Assuming we can't fix the gaming industry - and it's a fair bet that we can't - and assuming we can't fix the scoring system itself - it's a known problem and is being talked about - the only thing well-meaning supporters of their favorite games can do is to ask excessively critical number scorers to remove their score from the final equation.

This is fair. So long as someone is politely asking you to do something, even if you refuse or even if you take offense to their request, using this line:
"You've just been served in Jim's kitchen, prick!"
Is completely unjustified.

On a final note, to the good people at the Escapist: Please don't think my taking the time to type this is any positive indicator as to the quality of this series. Even if the goal is to get people riled up and cause controversy, there are better, more tasteful, more mature ways to go about it than the annoying character in Jimquisition. I decided to give it another chance after deliberately avoiding the series, and immediately regretted my decision. I'll do my best not to make that mistake again.
 

Cousin_IT

New member
Feb 6, 2008
1,822
0
0
Nerds like numbers. Nerds like making & playing computer games. Metacritic is full of numbers about computer games. Therefore nerds like metacritics numbers about computer games. Mystery solved.

Also: am a little bored of how The Escapists two main games "critics" both rely on coarse humour to make their point, as if the audience is incapable of understanding a discussion without a dick joke attached. Hard to shake the image of games being for children when the most public discussion around them rarely extends beyond school yard levels of eloquence.
 

briunj04

New member
Apr 9, 2011
160
0
0
freaper said:
<img src=http://www.google.com/recaptcha/api/image?c=03AHJ_VuvizGxQD_BUxRFIq5M7MirSEgWAyNHjTF0opTd6WftbztvmP3dEaAjW5qKTBejcpGR8kOXys_vVibXKlNLkCHjflpOARS-ixHSz5rHCr2QdFKv4toMsLTn60Kxk8yp--XrtrryaDxoQLG6jzw1FsnetOP_aGw>

(apparently Captcha now provides words that are upside down)
Heck, I got a Kanji symbol before! Whoever designed Captcha is crazy (-_-)
 

EvilestDeath

New member
Nov 4, 2009
115
0
0
Why does he keep bringing up god? I don't have a problem with it I am just wondering if it is an attempt to antagonize those who would jump all over that subject or if he is more genuine about it. I think it is humorous but I also don't want to take that statement in the wrong manner. Since his tone of voice and usage of the term seems like he is trying to get some negative responses. The broken yet common phrase "Not sure if troll" seems to apply here as my general statement.
 

Not G. Ivingname

New member
Nov 18, 2009
6,368
0
0
Jim Sterling said:
Metacritic Isn't the Problem

Oh you poor, sad little cretins. You are all so wrong. Always. Forever. Jim Sterling illuminates your path, but what good does that do when you refuse to open your eyes? Oh, he's so much better than you.

Watch Video
If you want to be taken a little more seriously, might I suggest you get your hands on a camera with better quality. I think half the haters will shut up if these videos were much less grainy.

You do have a have point with Medacritic, it is not to blame for anything just as a gun maker isn't at fault for a shooting, your video's are just irritating to look at compared to the "cleaner" looking ZP or EC.
 

CM156_v1legacy

Revelation 9:6
Mar 23, 2011
3,997
0
0
Woodsey said:
An old saying comes to mind... now what was it? Oh yes: "don't like it, shut the fuck up and watch something else."
Beggin' your pardon, but I have a question.

Are you saying that people (unlike the person you were responding to) who don't like the show have any less of a right to say what they don't like? I mean, to be fair, Jim IS a critic, isn't he? Turnabout is fair play and all that.

Besides, what do you care if people comment that they hate it?

OT: Great Jim. I just love your sense of humor. Or how you keep being sarcastic. I honestly hope you start by saying "Preps, stop flammin' the story"
 

erbkaiser

Romanorum Imperator
Jun 20, 2009
1,137
0
0
briunj04 said:
freaper said:
<img src=http://www.google.com/recaptcha/api/image?c=03AHJ_VuvizGxQD_BUxRFIq5M7MirSEgWAyNHjTF0opTd6WftbztvmP3dEaAjW5qKTBejcpGR8kOXys_vVibXKlNLkCHjflpOARS-ixHSz5rHCr2QdFKv4toMsLTn60Kxk8yp--XrtrryaDxoQLG6jzw1FsnetOP_aGw>

(apparently Captcha now provides words that are upside down)
Heck, I got a Kanji symbol before! Whoever designed Captcha is crazy (-_-)
You know what it stands for, right?
_C_ompletelty _A_nnoying _P_rogram _T_hat _C_auses _H_orrible _A_gony.
 

cynicalsaint1

Salvation a la Mode
Apr 1, 2010
545
0
21
Really Metacritic's biggest problem is this:

You can't simply equate one site's review score with another's. IGN really only uses the top of its 10 point scale for example, other sites use the whole range. The Escapist's system also doesn't convert over to the 100 point scale very well either.

I find trying to average all these scores that may mean different things in the context of their own system a bit disingenuous.

I do however find Metacritic useful for seeing how a game is doing in general, not by looking at its Metascore but through it giving me a quick look at the various reviews its collected. From there I'll usually look at a couple of the best and worst reviews and use that to figure out whether or not I'll go for the game.
 

Patton662

New member
Apr 4, 2010
289
0
0
The hate you get is hilarious.
I agree, metacritic is an example of how the games industry is using metrics wrong.
 

mjc0961

YOU'RE a pie chart.
Nov 30, 2009
3,847
0
0
Well, there's one thing I agree with this week: too many people put way too much stock in scores. I still think scores should just go away from all reviews though. They're too vague to tell me if a game is something I would enjoy or not; the full review is always needed. Either I check the score first and go "Well shit, that didn't tell me anything. Guess I'd better read the review.", or I read the review first and then get to the score and say "What do I need this for? I already know about the game after reading the review."

I don't know why both gamers and the industry focus on scores so damn much. It seems like the only real use for them is to make sure your reviews can get on Metacritic which gets you more attention from readers and publishers. That and keeping the stupid people who rely on scores too much happy, but as I'm not responsible for making sure a website gets enough traffic to stay in business, I'm more inclined to say let them suffer without scores.

In the end, ignoring review scores is easy, but ignoring all the idiots who constantly whine and piss and moan and cry about the score a game got (Duke Nukem Forever for example, or if anyone else remembers a few years back when GameSpot gave Twilight Princes an 8.8 and from the reaction you would have thought the world was going to end) is really hard. Maybe there needs to be some add-on for browsers that auto-scans forums and video content and if it's someone whining about a game's review score, it gets auto-hidden. Ah, what a wonderful internet that would be.

Woodsey said:
An old saying comes to mind... now what was it? Oh yes: "don't like it, shut the fuck up and watch something else."
For the love of all things, this. This is the 9th video of the series as far as its run on The Escapist goes. If you don't like it yet, you aren't going to, so quit watching every week just to flame the show and Jim in the comments. Go find something better to do with your time.

CM156 said:
Beggin' your pardon, but I have a question.

Are you saying that people (unlike the person you were responding to) who don't like the show have any less of a right to say what they don't like? I mean, to be fair, Jim IS a critic, isn't he? Turnabout is fair play and all that.

Besides, what do you care if people comment that they hate it?
As stated, this is the 9th video. People should have learned weeks ago that they don't like it and thus they shouldn't watch anymore. I think it's fair to tell the remaining people who watch knowing full well they won't like it and then come on here just to insult Jim to shut it and go do something else. Constructive criticism I have no problem with, but come on, is it necessary to watch and comment "I hate this I wish Escapist would remove it" every week? If they really want it to go away they should just stop watching entirely anyway, because not enough views is what kills a show, not people watching it and bitching about it.
 

Lullabye

New member
Oct 23, 2008
4,425
0
0
I've never even been on meta critic before. Is it really that big a deal? Do publishers really think that they will sink or swim if they don't get a decent score on the site? I can say that as a consumer, reviews. especially teh 1-10/10 kind don't mean crap to me and do not affect my purchases.
 

Birthe

New member
Apr 26, 2010
73
0
0
Have to say part of the fun of watching this is reading some of the comments later on by people hating it so much, so great comment on this early in the video.

Some great comments in the video, from this one and some other things said on here especially also by yahtzee you can really get the feeling that publishers are lately the main thing that seems to be wrong with the gaming industry.
 

Woodsey

New member
Aug 9, 2009
14,553
0
0
CM156 said:
Woodsey said:
An old saying comes to mind... now what was it? Oh yes: "don't like it, shut the fuck up and watch something else."
Beggin' your pardon, but I have a question.

Are you saying that people (unlike the person you were responding to) who don't like the show have any less of a right to say what they don't like? I mean, to be fair, Jim IS a critic, isn't he? Turnabout is fair play and all that.

Besides, what do you care if people comment that they hate it?
Its 9 in videos in; there's been plenty of time to whine and complain. People won't like everything on the site - they don't venture in hordes onto the other stuff to write comments that aren't even critical of the actual piece, but simply the show's existence.
 

mjc0961

YOU'RE a pie chart.
Nov 30, 2009
3,847
0
0
Lullabye said:
I've never even been on meta critic before. Is it really that big a deal? Do publishers really think that they will sink or swim if they don't get a decent score on the site? I can say that as a consumer, reviews. especially teh 1-10/10 kind don't mean crap to me and do not affect my purchases.
Yes they do, because unlike you, me, and other people, some gamers won't buy a game if the Metascore is "too low."

And usually their idea of "too low" is something insane like 89 or under.
 

CM156_v1legacy

Revelation 9:6
Mar 23, 2011
3,997
0
0
Woodsey said:
CM156 said:
Woodsey said:
An old saying comes to mind... now what was it? Oh yes: "don't like it, shut the fuck up and watch something else."
Beggin' your pardon, but I have a question.

Are you saying that people (unlike the person you were responding to) who don't like the show have any less of a right to say what they don't like? I mean, to be fair, Jim IS a critic, isn't he? Turnabout is fair play and all that.

Besides, what do you care if people comment that they hate it?
Its 9 in videos in; there's been plenty of time to whine and complain. People won't like everything on the site - they don't venture in hordes onto the other stuff to write comments that aren't even critical of the actual piece, but simply the show's existence.
But that still begs the question of why you at all care? Really, if people leave negative comments, even if they are ill informed, how do they affect/bother you?