Jim Sterling said:
Metacritic Isn't the Problem
Oh you poor, sad little cretins. You are all so wrong. Always. Forever. Jim Sterling illuminates your path, but what good does that do when you refuse to open your eyes? Oh, he's so much better than you.
Watch Video
By and large, I agree with what you're saying. I'd even go so far as to say your knife simile is inaccurate. This isn't like blaming knife makers for stabbings. This is like blaming a hammer maker when someone tries to use the hammer as a shovel.
But I also think that MetaCritic, recognizing the potential hazard of the tool they've created, could choose to do things a bit differently to mitigate those ill effects. Even gun manufacturers put "safeties" on their products, don't they? Taking responsibility for a solution isn't the same as taking blame for the problem.
You rightly noted that MetaCritic could use some more transparency. Letting people see a bit further into the process might allow them to make a more informed interpretation of the result. Otherwise, it's giving people an answer without really telling them what the question was. They'll create their own questions, and apply them in whatever way best suits their desires at the moment.
Of course, this also requires
scoring reviewers to be more transparent about what their numerical or letter scores actually mean. For one reviewer, a 7/10 might be a near-failing score (like a D in school). For another, a 7/10 might be phenomenal, as a 10 represents some perfect super-game that hasn't been created yet, and 70% of that ain't bad.
For many reviewers, the difference between each "step" on the scale decreases as you move up it. This is especially true of scales of 10 points or larger. Others, usually those that employ "five stars" or so, try to make each step a bit closer to symmetrical. That makes comparing and "normalizing" different scores a lot harder. I'm sure some of the folks behind a MetaCritic score are trying to take this into account, but a bit more openness on both sides would help us as consumers decide a bit better.
Of course, ultimately, the problem does rest with publishers themselves. The reason we weigh game reviews so differently from movies and music is simple: games are
far more expensive. I can see a movie for $9, in its entirety. From that, I can also decide if it's worth the $20 to get it on DVD, or maybe $30 for some added features. That means, for any given movie, I can try the complete product
and then purchase it, if I so choose, for
half the price of a video game.
With a game, "trying" means "buying." More than with movies, we rely on others to "try" the game for us. And for the prices being asked, anything less than a
glowing review is going to be seen as a "wait and buy it used" recommendation (and by the time that's feasible, the game has usually been forgotten in the hubbub of some other shiny new release). Until publishers can be swayed from the $60 price tower, that's not going to change.
Consumers are wholly justified in the weight they put on reviews. Publishers are justified in the weight they give to reviews as well, but they are
not justified in their response to that assessment.
Reviewers (and MetaCritic) are intended to serve as a go-between. They are meant to communicate the merits (or faults) of a product to the consumer so that we can make good decisions,
and so that both reviewers and consumers can communicate a message about our expectations to the publishers.
Consumers can
only communicate via numbers (dollars and copies), which is about as fine-tunable as a game of "Marco Polo." Reviewers can communicate via narrative... but instead, we're allowing that narrative to be boiled down into
just numbers, with no real context.
When communication breaks down, it doesn't matter whose
fault it is. The sender has the
responsibility to adjust the message until it is received clearly. MetaCritic might be distilling the message down a bit too much, to the point that the message is being sent
louder, but not any
clearer.
As a teacher, I know full well that sometimes students don't learn a concept because they're lazy or not trying hard enough, and sometimes it's because I didn't express it as clearly as I should.
But in both cases, the responsibility is on me to be the one that says it differently or changes the stimulus until I get the result I want. So, yeah, the publishers are using it in the wrong way. They're reading the message the wrong way. Their receiving feedback in the wrong way. But they're not going to change first.
TL;DR: MetaCritic is being misused, and it's not their
fault. However, knowing that they're being misused gives them a certain amount of
responsibility to help change that. So while it's unwise to throw the baby out with the bathwater, it's equally unwise to just
leave the baby in the bathwater. It's not unreasonable to ask MetaCritic to change how they do things.
