The problems with the supposedly "unbiased" review

[Kerethos]

I keep seeing, and hearing, people talk about the call for unbiased reviews. But really, there can be no such thing from any human being that has ever played a game or has any opinions on anything. At least if you want an honest review.

Now, for sure, I don't think you have any business reviewing a game made by a friend or someone who you have certain financial ties to (such as where you stand to gain financially from the success of the game) or when you have worked on creating the game. That kind of bias through relationship is easy to identify, and avoid, and is generally considered as being corrupt - rather than just biased.

But let's get creative with an example of personal, rather than financial or relationship based, bias:

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

Should I disregard my experience and judge the game wholly on its mechanics and execution, or should I take my experience into account and rate it based on how I experienced it - meaning based on my own values and enjoyment?

I think cases like these illustrate the problems with assigning a numbered score; as scores are problematic because it's often all people look at, rather than the actual review. Without scores the actual review does, in my opinion, carry more weight.

But as it is, ultimately, it's the number assigned at the end that carries actual weight.[footnote]I do not review games, but if I did I would not assign them scores. I'd possibly assign them arbitrary things like: "I give this game 4 penguins wearing funny hats, 25 sad seals and one seal whisperer - there to help cheer up the sad seals and teach them how to love again." or "One potted plant and a companion pillow with the print of your favorite banana, half peeled."[/footnote][footnote]Captcha: that will not work. Shut up captcha, I'll give whatever random nonsense that pop up in my head at the time as a hypothetical scores if I want to.[/footnote]

 

[BathorysGraveland2]

Honestly, completely unbiased reviews sound about as enjoyable as watching paint dry. If a reviewer is allowed to show his excitement, and his passion toward something, then it can lead to far more creativity and will be more enjoyable to watch/read as a result. Force someone to give a boring, unbiased and "professional" review on something, and keep certain emotions locked up, would make for something even more disingenuous, in my opinion.

I'd much rather reviewers speak openly and honestly.

 

[sanquin]

I've been saying this for years already. There is no such thing as an unbiased review as long as a human being made it. You can only judge someone based on how you experienced it. Nothing to do about it. Doesn't mean you can strive to be unbiased, but you can never get rid of it.

 

[Savagezion]

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

Should I disregard my experience and judge the game wholly on its mechanics and execution, or should I take my experience into account and rate it based on how I experienced it - meaning based on my own values and enjoyment?

I think cases like these illustrate the problems with assigning a numbered score; as scores are problematic because it's often all people look at, rather than the actual review. Without scores the actual review does, in my opinion, carry more weight.

But as it is, ultimately, it's the number assigned at the end that carries actual weight.

You should rate a game like that highly. Good mechanics, good production value, well optimized, as well as a story that holds up? Sounds like a winner to me on all fronts. Now, for the sake of argument, the narrative in the game supports divorce and this sits sour with the reviewer who believes divorce is evil and so he gives the game a "6" or a "4" saying it "ruined the game" for them. The problem is you are now not reviewing game design but instead reviewing moral principals. Earthbound is often cited as having negative messages about capitalism in it. If a reviewer dings the game for that, they are actually reviewing political ideologies rather than game design.

When I read a review of a game, I want to know what works and what doesn't in the game. I want to hear about the combat model, any leveling system present, diversity in playstyles offered, etc. Many people have varying outlooks on game design. Some people love games about grinding, others don't. Some people love looking for hidden collectibles in a game, some don't. Some people hate stories told through cutscenes, some don't. This list could go on for miles as well as variations of the things I have listed like some people prefer the collectibles to grant xp, some dont - and so on. The reviewer will have opinions on things like this and I would much rather hear them discuss these things rather than how big someone's tits are or how the game preaches the "wrong" message and their feelings about things like that. I don't care, I wanna know how good the game is.

This is actually where number scores should have a purpose. The game you describe is worthy of an 8,9, or 10. However, in the review, you could explain that it is a well designed game but "factor X" ruined it for you, personally. By doing so you admit that the game itself isn't bad, but that thing that bothers you is something you couldn't get past but others may. At which point the reader can decide for themselves instead of you trying to convince them why they should agree with you.

Reviewers have the freedom of press to push ideologies in their reviews but a lot of us are getting sick of it and just want them to review the actual game design, not the politics. Bias will still exist but it will be bias as to what types of games and mechanics a reviewer likes not whether they lean left or right.

 

[stroopwafel]

Well, I think it's impossibe to objectively quantify something as personal as enjoyment. The only thing a 'professional' reviewer can do is critically discern why he or she thinks a game is either good or bad. And even then, as long as a game is technically sound(which most modern games are) even that is highly subjective. That's why I think you should always try a game for yourself and not just go by reviews/general (internet) consensus. I agree the numerical score at the bottom of a review oughta go, but that's not going to happen.

 

[Silvanus]

An objective review would be a statement of fact-- it would scarcely qualify as a review at all.

It's the difference between "The writing is awful, whoever wrote it has no idea how english-speaking human beings actually speak to eachother, the translation made me cringe." and "The dialogue offended me and did not fit with my world view, therefore I'm going to mark it down."

Neither of those are objective.

It's the difference between "Whoever designed this character was a clown, seriously learn what colours compliment eachother and try again." and "Skimpy outfits, you say? Unnaceptable."

Nor either of those.

It's the difference between "They failed to give the main character one thing a character should have and that is character." and "I don't like the main character being a straight white male."

Nor these.

In fact, none of those you mentioned were objective. The actual difference is that some of those you mentioned contained criticisms you deem valid, and others don't... but your own criteria, by which you judge that, are also subjective.

(Not to mention the last example I quoted was, of course, a strawman).

 

[Scootinfroodie]

I think what it boils down to is eliminating CoI's/perception of CoI's, being honest about biases, not allowing one's personal politics/interpretation to negatively affect the stated quality of the game (I'd suggest that a second piece be written for these. e.g. Tropico 5 - is the game any good and Tropico 5 - why couldn't the game make me feel nice about being a dictator?) and possibly even knowing the subject matter.

The first bit should be obvious.

The second/third bit is basically what should have happened with Polygon's Tropico review as I not so subtly suggested previously. One should consider the game and its mechanics and leave political ideology and "muh feels" for another article. If the entire point of a game is showing the user the fine line one can be forced to walk when given nigh unlimited power over another person, and the ease of sliding into corruption, then docking points because you don't *like* corruption is silly. If anything, your lack of comfort is an example of the message of the game effectively being delivered. I also find it hilariously ironic that in this era of "we want games that make us think" a game that makes you think of something outside of the standard cozy whitewashed set of social concerns is scary and worth rating lower. I wonder if even Revengeance would have become an "uncomfortable" title if reviewers had managed to get past Blade Wolf, despite the ridiculous delivery of its message.

For that last bit, reviews of basically any fighting game, spectacle fighter, any comp shooter that isn't CS etc. are generally done by someone who doesn't play those titles outside of their mandated review session, leading to really poor surface level praise and complaints. I remember seeing a write up on RPS about Skullgirls where the reviewer complained about the characters without having played through their stories (outside of like.. 2), attacked the artstyle, and then whined that the tutorials didn't make him not suck against regular fighting game players online. At no point were the actual mechanics addressed because he didn't really know anything about fighting games. While its valid to make note of newcomer-style complaints/perspectives, you're ultimately reviewing your personal level of dedication and not the game. As someone who is payed to constantly play a stream of new titles, your level of dedication is going to be relatively low by default.

 

[Bombiz]

i've kinda just stopped reading/listening/watching video game reviews for actual information. Now i mostly read/listen/watch them for fun/entertainment. i've always just went and looked up gameplay footage/LP's online to get a good grasp of the game and what it's about.

 

[Smooth Operator]

It is absolutely true that a human is not capable of objective reviews, our entire perception of reality is based on subjective interpretation of the information our brain takes in.
But what a professional can/should do is understand his biases and rather then emotion pass on the conflicting information, and cover both sides of an issue. A review saying "I love/hate the AI" tells us fuck all about the game and only really describes how the reviewer felt at the time, describing the advancements or shortcoming of X feature however takes their bias mostly out and the viewer decides if this is problematic for them.

This is where the distinctions are made between professionals, entertainers, personal views, and political pieces... they may all call it the same, but it is very far from it. Sadly this medium is still not at a stage where review standards would get properly established and distinctions made based on that.

 

[Silvanus]

There's a difference between setting aside your personal bias and forming an opinion without your ideology cropping in and simply listing off things you don't like.

Both of those things are subjective. Forming an opinion is, in itself, not being objective.

It is possible for a translation to be horrible and not flow into english, particularly in Eastern European games, objective fact.

It is possible to make a character that looks horrible because the designed colours do not compliment eachother, objective fact.

No, it is not. "Horrible" is subjective. "Not flow" is subjective (unless we're talking about physical solids). "Do not compliment" is subjective. They are all expressions of opinion.

It is possible for a character to be bland and insipid, objective fact.

It is impossible for a game to have bad writing because it offended you, subjective opinion.

It is impossible for a game to have bad character design because cleavage, subjective opinion.

It is impossible for a game to have a bad character simply because the character is a straight white male, subjective opinion.

I find it incredibly odd I actually had to explain the difference between a fact and an opinion, this is 2014.

No, no, no. Nothing you have listed is actually objective. You're misusing [https://www.butte.edu/departments/cas/tipsheets/thinking/claims.html] the [http://www.diffen.com/difference/Objective_vs_Subjective] term [http://www.merriam-webster.com/dictionary/objective].

 

[Scootinfroodie]

Silvanus, it feels very much like you are nitpicking. There are ways to objectively measure the efficacy of a translation. English certainly has a lot of flexibility, but there are predefined rules one can reference. If I'm using the wrong words, poor grammar, misspelling words etc, then the translation is of lower quality. "Horrible" is a term that reflects the low quality, though possibly in a hyperbolic fashion. Similarly, a game being buggy or unstable can be assessed using a clear metric. When it comes to less clear-cut topics, disclosure of bias and a demonstrable attempt to separate oneself from it ought to come before one's pet causes and keyboard crusades

 

[WindKnight]

The nearest I've seen to an 'objective/unbiased' review was one of Prince Of Persia: warrior within, where the reviewer gave it a 7 out of ten as thats how 'mechanically good' the reviewer felt it was... but the entire review dripped with language and sentiment that made clear just how much he loathed the game for basically crapping on everything that had made Sands of Time so wonderful.

 

[Worgen]

I can tell you exactly how to make an unbiased review. Just say what the controls are and call it a day.

 

[Mangod]

I'm just gonna leave this revelation from the man, the prophet, the God himself, Jim Sterling, here for your perusal:

http://www.destructoid.com/100-objective-review-final-fantasy-xiii-179178.phtml

 

[sageoftruth]

My take, some reviewers are too egotistical.
Many seem to be forgetting they're there to discuss the product not try to push or tell people to care about their own personal politics. Should I keep inserting getting more women into science or attempting global warming or world overpopulation into all my reviews ?

Just because truly objectivism is near impossible doesn't mean you can go "oh fuck it I won't bother" it's like saying don't strive to make a perfect game because it's impossible so why bother ?

Games are a big enough subject to be able to be talked about on their own without tacking on other subjects and doing both a dis-service.

That's a good point. Striving for objectivity is important, provided that's what the reviewer is shooting for. I'd never ask Yahtzee to change how he does his reviews, but I'd be pretty disappointed if TotalBiscuit suddenly decided "Screw it" and decided to imitate Yahtzee. I think having both kinds of reviewers is the best possible outcome.

Also I wouldn't point this out if it wasn't hilarious (apologies in advance) but you'll probably want to go to the Ayn Rand thread if you want to talk about true "objectivism" as you put it. Lol. No offense.

 

[Scootinfroodie]

That's not nitpicking, you clearly don't know what objective means. If something is "of poor quality", then that is relative is everything else. Now no one has actually experienced everything else, so they're basing it of on their own measurements, which are very different from person to person.

You're trying to turn this into an argument of semantics. The point is that there are objective measures (how the English language is structured) and things that can be measured (how a translation fits that structure). The infamous "People die when they are killed" line is an amusing, but poor example of a proper translation.

If a critic of classic literature were to play, say, the Last of Us without having played any other game ever, he'd probably remark on how poorly written and terribly structured it was. If some redneck that hadn't experienced anything more complicated than American Gladiators, they'd probably say it was too weird and boring.

How are these related to what I mentioned previously? Additionally, "I don't play many games" or "I don't really enjoy these kinds of games" is a simple method of ensuring that your audience is aware of what may potentially be a flawed analysis

Some people (like me) thought Inception was incredibly simplistic to a fault, while a lot of other people thought it was too complicated to a fault, while another group of people thought it was pretty good in that regard. No one there is right, you just compare it to similar experiences.

We're not discussing whether or not you liked a movie, we're discussing the methods for credible critique and product reviews. If someone likes DA2 but explains in depth what the pros and cons were, the reader can come to a reasonably accurate conclusion as to whether or not they will enjoy the product. If someone complains because they didn't like the voice acting, but doesn't really explain that reason or how it invalidates the actual gameplay/story, or decides the whole game is actually about destroying america and writes a scathing article condemning their own interpretation, then the reader is left largely uninformed.

Except a review is not a fucking message board. A review is there to that the reviewer can talk about their own personal experiences with the product. If a game, say, insinuated that gays were Satans hellspawn and deserve to die and it's your civic duty to kill them, I'd feel very bothered by that and it'd probably make my experience more negative.

If the reviewer insinuated that every game was about attacking the (insert group here) community, would you be more or less inclined to believe and promote them? If they started insinuating that the act of creating or controlling a character implied harm, would you deem them someone who can accurately and calmly discuss a game's strengths and flaws?

Remembder, just because some person thinks that a female character wearing high heels in a combat situation is incredibly offensive and terrible and ruined the game for them, why do you care? Their opinion obviously shouldn't apply or get through to you, sicne it didn't bother you. A review isn't a circlejerk where everyone agrees what was good about the game or not, it's nothing mroe than the reviewer writing down what they thought about it. "Star wars is for teh gheis lol" is still an actual review, though not one you'd probably care about. just because some reviews are longer and more proofread, doesn't mean their opinion is somehow more valid.

You do understand that if a large number of reviewers rate down a game for high heels, the developers can literally lose millions for one single aspect of a character model. I'm not suggesting that metacritic bonuses SHOULD exist, but it's important to realize they do

Additionally, we're not talking about someone's poorly written comment on the internet, we're talking about people who expect a certain level of respect and viewership. If they haven't the slightest respect for their audience, then they shouldn't have one. That's why people are fed up with Polygon/RPS/etc.

 

[Skull Bearer]

... which is why you should more than one review to see if the bad notes are shared by more than one person. If you all want bleating sheep parroting the same lines, you're doing the medium a lot of damage.

Hell, you've already done the medium a lot of damage. Gamergate's resulted in the loss of several developers and so much bad press it'll take years to correct. What a miserable lot.

 

[Shingeki no Gingerbeard]

A 100% unbiased anything in Journalism, or something approaching it? Yeah, not going to happen. The best you is try to be as impartial as possible and be upfront about yourself and your background so people know where your colors are coming from when they do show.

 

[Verlander]

It is possible for a translation to be horrible and not flow into english, particularly in Eastern European games, objective fact.

It is possible, but I'd question the "particularly in Eastern European games". They're not exactly a major offender of anything.

It is possible for a character to be bland and insipid, objective fact.

Bland and insipid are opinions.

It is impossible for a game to have bad writing because it offended you, subjective opinion.

If a game has not set out to offend you, then that is a pretty good indicator of bad writing.

It is impossible for a game to have bad character design because cleavage, subjective opinion.

If the character design is offputting or offensive to potential customers because of said cleavage, that is bad design. Objective fact.

It is impossible for a game to have a bad character simply because the character is a straight white male, subjective opinion.

Never been a problem.

I find it incredibly odd I actually had to explain the difference between a fact and an opinion, this is 2014.

I find it odd that you think that objective opinion is a more worthy critique of something. Games aren't parts of machinery to complete a singular task, they are experiences designed for people.

 

[spartan231490]

You missed the most important point. Games, in fact all art, is a subjective experience. An objective review is meaningless since were are not looking for an objective experience but a subjective one.