The problems with the supposedly "unbiased" review

[Something Amyss]

Just be aware that it doesn't necessarily debunk any claims if some random internet goer makes a false claim of bias

The hell are you talking about?

 

[veloper]

In it there are several types of bad guys, goombas, koopa troopas, bullet bill and more. The main bad guy is Bowser, who you will fight multiple times.

You are teh BIAS! Who are you to say that Bowser and the koopas are bad guys?! Do you really hate turtles or something?

I don't think the TS was calling for the removal of all subjectivity.

 

[GloatingSwine]

Think you may have misunderstood my comment. My complaint wasn't "I want these people silenced", it was that no single school of thought should control all reviews of culture or art, nor should adherents to a single school of thought, in positions of editorial control, collude in a clandestine manner to aggressive force an agenda. It's unhealthy, no matter how much one happens to agree with the school of thought, you just end up on an unpleasant witch-hunt, single issues start dictating review scores, people start competing to write the most "zeitgeist-y" review and ultimately you end up with useless reviews. Nor should people be hounded from their homes or abused due to their agenda. The issue, to my mind, it's binary, there's quite a lot of people doing things that don't help.

Yes, lots of people imagining conspiracies of reviewers trying to aggressively push particular agendas, and that doesn't help.

Because you have literally no way to tell whether it's the Ebil Feminist Conspiracy (because don't even fucking pretend you're really worried about the Rotarians controlling gaming or whatever the shit) or that those reviewers actually honestly stand by what they wrote.

 

[Scootinfroodie]

Just be aware that it doesn't necessarily debunk any claims if some random internet goer makes a false claim of bias

The hell are you talking about?

Except your former examples are prevalent within the reviews people accuse of "teh bias."

IE if it's an isolated case of a small group of people attacking a review for the wrong reasons and then labeling those reasons "bias" that doesn't detract from the original claim. Granted, there's also plenty of room for misappropriating claims. That's why it's important to have a discussion and not a snark-fest

To indulge a little myself though, I don't suppose you plan on actually engaging any points, do you

 

[veloper]

In it there are several types of bad guys, goombas, koopa troopas, bullet bill and more. The main bad guy is Bowser, who you will fight multiple times.

You are teh BIAS! Who are you to say that Bowser and the koopas are bad guys?! Do you really hate turtles or something?

I don't think the TS was calling for the removal of all subjectivity.

Well that's the thing, who are we to limit subjectivity? If they fall in the guidelines of proper critics rules, meaning like a debate you show up with proper notes and reasoning to back up your review, then they're doing their review right. We don't have to like the review (goodness knows I've read reviews I certainly don't agree with), but it's their right as a reviewer.

But calling for reviewers to be more objective is limiting their right to be critics of their genre. And if that's the case, then we're heading to something a bit more like what I wrote above. We either have to understand the idea of free speech rights, the right to be a reviewer, the idea of good reviews vs bad reviews, and how to find a reviewer/reviewer site that we like, vs the objectivity of a review and how dead and hollow it would sound.

If that makes sense.

I reckon there's a bit more to it than that.

The critic can of course say whatever he likes, but the readers can say whatever they like about the review. I'm all for free speech.
That still doesn't help much though. So what makes a good critic?

I reckon for a critic to be a good critic, they should be able to cater to a wider audience than their circle jerk of ten close friends. And that I think requires a broader perspective.
I think a really good critic could answer the question: who is this game for?

 

[Something Amyss]

IE if it's an isolated case of a small group of people attacking a review for the wrong reasons and then labeling those reasons "bias" that doesn't detract from the original claim.

And so I have to ask: so what?

I'm not even sure you're actually engaging me. I wasn't sure from the start, to be frank.

 

[Fdzzaigl]

A review is never unbiased.

However, it should contain a healthy balance of opinion and facts about the game so that the reader can judge whether the product is something he / she would enjoy.

If people don't understand that, they simply don't know what a review is.

 

[veloper]

In it there are several types of bad guys, goombas, koopa troopas, bullet bill and more. The main bad guy is Bowser, who you will fight multiple times.

You are teh BIAS! Who are you to say that Bowser and the koopas are bad guys?! Do you really hate turtles or something?

I don't think the TS was calling for the removal of all subjectivity.

Well that's the thing, who are we to limit subjectivity? If they fall in the guidelines of proper critics rules, meaning like a debate you show up with proper notes and reasoning to back up your review, then they're doing their review right. We don't have to like the review (goodness knows I've read reviews I certainly don't agree with), but it's their right as a reviewer.

But calling for reviewers to be more objective is limiting their right to be critics of their genre. And if that's the case, then we're heading to something a bit more like what I wrote above. We either have to understand the idea of free speech rights, the right to be a reviewer, the idea of good reviews vs bad reviews, and how to find a reviewer/reviewer site that we like, vs the objectivity of a review and how dead and hollow it would sound.

If that makes sense.

I reckon there's a bit more to it than that.

The critic can of course say whatever he likes, but the readers can say whatever they like about the review. I'm all for free speech.
That still doesn't help much though. So what makes a good critic?

I reckon for a critic to be a good critic, they should be able to cater to a wider audience than their circle jerk of ten close friends. And that I think requires a broader perspective.
I think a really good critic could answer the question: who is this game for?

I would say most critics do that. Jim Sterling does, Total Biscuit does, Yahtzee does. Hell even IGN, who I can't stand, does. I mean, they pander to the fan base of games. They're going to sell games to people that already planned on buying the games in the first place.

Total Biscuit can do this to some fair degree, but I'm not so sure about the other guys.
Yahtzee really shouldn't have reviewed JRPGs and I haven't clicked on IGN for a long time, because all their big pieces sound like pure advertisement to me.

I'm not quite sure what that has to do with objectivity, I'm sorry. Perhaps I'm not seeing where you meant to go with what you were saying, but I'm not sure what you mean in relation to the unbiased review.

I don't think anyone can see that wider perspective, without being able to look past their own biases and personal opinions first.

 

[Scootinfroodie]

IE if it's an isolated case of a small group of people attacking a review for the wrong reasons and then labeling those reasons "bias" that doesn't detract from the original claim.

And so I have to ask: so what?

I'm not even sure you're actually engaging me. I wasn't sure from the start, to be frank.

Was just noting it, and your continuing evasion. For someone who is fully willing to make all sorts of claims, you seem rather averse to any sort of explanation or evidence.
To bring things back on topic

You made two claims,
one was that people were claiming bias despite the methods outlined previously being used
the other was that the remainder of the claims were "conspiracy theory territory"
Do you have any supporting evidence or argumentation, or did you just expect people to accept your assertions as given?

 

[veloper]

In it there are several types of bad guys, goombas, koopa troopas, bullet bill and more. The main bad guy is Bowser, who you will fight multiple times.

You are teh BIAS! Who are you to say that Bowser and the koopas are bad guys?! Do you really hate turtles or something?

I don't think the TS was calling for the removal of all subjectivity.

Well that's the thing, who are we to limit subjectivity? If they fall in the guidelines of proper critics rules, meaning like a debate you show up with proper notes and reasoning to back up your review, then they're doing their review right. We don't have to like the review (goodness knows I've read reviews I certainly don't agree with), but it's their right as a reviewer.

But calling for reviewers to be more objective is limiting their right to be critics of their genre. And if that's the case, then we're heading to something a bit more like what I wrote above. We either have to understand the idea of free speech rights, the right to be a reviewer, the idea of good reviews vs bad reviews, and how to find a reviewer/reviewer site that we like, vs the objectivity of a review and how dead and hollow it would sound.

If that makes sense.

I reckon there's a bit more to it than that.

The critic can of course say whatever he likes, but the readers can say whatever they like about the review. I'm all for free speech.
That still doesn't help much though. So what makes a good critic?

I reckon for a critic to be a good critic, they should be able to cater to a wider audience than their circle jerk of ten close friends. And that I think requires a broader perspective.
I think a really good critic could answer the question: who is this game for?

I would say most critics do that. Jim Sterling does, Total Biscuit does, Yahtzee does. Hell even IGN, who I can't stand, does. I mean, they pander to the fan base of games. They're going to sell games to people that already planned on buying the games in the first place.

Total Biscuit can do this to some fair degree, but I'm not so sure about the other guys.
Yahtzee really shouldn't have reviewed JRPGs and I haven't clicked on IGN for a long time, because all their big pieces sound like pure advertisement to me.

I'm not quite sure what that has to do with objectivity, I'm sorry. Perhaps I'm not seeing where you meant to go with what you were saying, but I'm not sure what you mean in relation to the unbiased review.

I don't think anyone can see that wider perspective, without being able to look past their own biases and personal opinions first.

Yahtzee already knows he doesn't like JRPGs, but he has a job to do. He has to do it. The Escapist doesn't pay him to avoid JRPGs, or else I imagine he probably wouldn't play them. He's said as much in his own videos. And even when he does play them, he still manages to find things to praise about them, whether it's graphics, or something of the like. He may hate the game, but the fact is he comes with notes prepared on why he hates it.

He may have had to do them(maybe) but that doesn't make them good. Those pieces just ended up being uninformative. People who hate JRPGs will avoid the games anyway, while JRPG fans will want to know how a JRPG compares to the games they love.
I think that is not the way how to do it.

IGN is nothing but an advertising site, and that was my point. It is mostly for fanboys/fangirls. They're not writing reviews for everyone, they're writing them for the people they know is going to buy the next Halo, or the next COD, or the next whatever.

In which case you might as well not read the "review" at all, but just pre-order the game blind.
IGN is a good example of uninformative crap.

Because that's who their market is. And it's a big market. Just like Yahtzee's market is a pretty decent sized market. Just like Sterling's market is a decent, and so it TB's. These guys found a niche appeal, and they're good at what they do. That's how critics work. Not everyone liked Roger Ebert, some preferred Gene Shallot, but that doesn't mean that Roger Ebert didn't have a good market.

Specialization or a big following isn't what makes a critic good.
The size of the audience is only some indication. Specialization could actually be making it easier for yourself. Whatever thing comes before that is more interesting.

And the fact is, everyone who does a game criticism already knows they have a bias. You can't go into a criticism without knowing that. You can't sit there and say "I'm completely without bias about this game!" You go into writing a review knowing who you are when you write. And if you become a niche writer, as most editorial writers/critics become then it's going to become pretty apparent pretty quickly what kind of person/writer you are. And that may not be for everyone. You may only have a small corner market, or you may end up dominating a market. It just depends on your appeal. But that's how niche markets work.

I think knowing what you like or dislike is only a first step. Knowing who you write for and how you can best inform them is where the value is.

 

[MrHide-Patten]

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

This is exactly the kind of point I was arguing with somebody about the 7.5 review for Bayonetta, because reviews matter in this day and age. Really why do we even rate Nintendo games anymore, advertising reasons?

 

[MrHide-Patten]

very True i'd not ask Yahtzee to change because the review is a minor component in what is in reality a comedy work of entertainment.

Ho snap, ain't this just a delicious example of dismissal of critique. "Oh look at this person picking apart a game I like, oh they're just taking the piss, ho ho ho, arguments invalid".

I'm just gonna ditch all pretense here and say that this sort of rationalization; i.e. "comedy" or "entertainment" can't be an objectionable critique, pisses me the hell off. I wish that some people could take that not everybody likes what you like, and can be liberally taken the piss out of and that does not make the observation any less valid.

 

[Something Amyss]

Was just noting it, and your continuing evasion.

About that evasion thing. Yeah, I do get sort of evasive when someone picks peculiar wordings after ignoring context. It looks very much like a trap so I try to clarify before saying anything.

You're not helping with that. You're simply making this look more and more like you're trying to set me up for a "gotcha" response.

Accusing me of not wanting to respond to you when you're not making it very prudent for me to respond is an interesting take on the self-fulfilling prophecy.


Edit: And considering how much time I've already spent trying to explain to #gamergate that saying "nobody should be compared to ISIS" isn't a statement that I endorse comparing Gamergate to ISIS, I'm particularly suspicious of people I think are trying to retcon what I say.

 

[Antonio Scott]

I think that's why people give CoD high reviews score every year. Because for all intents and purposes. Its a good game. Every iteration feels like the last but it isn't a bad game is it? I think people need to re evaluate what will define the rating for a game. Accounting in other things aside from the obvious graphics, gameplay stuff.

 

[Scootinfroodie]

About that evasion thing. Yeah, I do get sort of evasive when someone picks peculiar wordings after ignoring context. It looks very much like a trap so I try to clarify before saying anything.

You're not helping with that. You're simply making this look more and more like you're trying to set me up for a "gotcha" response.

Accusing me of not wanting to respond to you when you're not making it very prudent for me to respond is an interesting take on the self-fulfilling prophecy.

Edit: And considering how much time I've already spent trying to explain to #gamergate that saying "nobody should be compared to ISIS" isn't a statement that I endorse comparing Gamergate to ISIS, I'm particularly suspicious of people I think are trying to retcon what I say.

I understand that you've had difficulty in the #gamergate thread (I've been monitoring it, and the previous thread. I've posted a grand total of... twice I think? and it was more to prod people back on track when I felt productive discussion was slowly being avoided intentionally or otherwise. I feel like I should be prodding more but don't want to make a habit of it). The environment that has a small handful of people responding to a large number of complaints/counterpoints/etc. can understandably be frustrating and confusing for both parties.

However, I don't feel that going through something in another thread gives you carte blanche to make claims and then not back them in this thread. The fact of the matter is that the first point is one that ultimately requires more discussion and less in the way of ultimatums. I think discourse on the methodology and quality of video game journalism and critique ought to be an ongoing process, no matter which way the latest controversy ends up going. The second point, however, comes across as nothing more than a shaming tactic, particularly when you consistently avoid explaining it.

To make things clear on my end, I don't see anything "conspiracy theory" worthy about suggesting that some reviewers might dislike a game for having a standard straight white male protagonist, or suggesting that reviewers might omit something they haven't got much experience in. As a clear-cut example of the latter, in the RPS Skullgirls Review most of the review boiled down to tutorials and skimpy outfits. The mechanics were largely untouched and the reviewer's comments about losing in multiplayer due to the inefficacy of the tutorials revealed that he was most definitely not someone who was familiar with fighting games and their mechanics... and I say this as someone who is pretty awful at all things fighting game

I think that's why people give CoD high reviews score every year. Because for all intents and purposes. Its a good game. Every iteration feels like the last but it isn't a bad game is it? I think people need to re evaluate what will define the rating for a game. Accounting in other things aside from the obvious graphics, gameplay stuff.

CoD reviews, at least from what I've seen, have actually been gradually lowering due to a lack of innovation. Basically if you keep your game the same and everyone else tries out new things, your game lowers in relative quality and people start to get bored. It's not to say that it's a bad game per se - it's a competently made franchise, but it's not exactly mind blowing to play through the same sequence of "something blows up -> you fall over -> canned animation where you are picked back up -> rollercoaster of gunsplosions" 10001 times. As for the MP, it's intelligently designed but until more recently there weren't many attempts to try to change things up in a significant way, and the game isn't really designed for long term play like CS, Starcraft, Quake etc.

 

[Kerethos]

Wow, this got a lot more people talking than I expected - quite the read so far.

I suppose it's time I inject my own actual opinions on the subject at his point.

First of, on the subject of an objective review, I think there's been a few good examples of such given. And they are entirely useless, essentially just a description of the game without any evaluation on if it's actually enjoyable or not. It's like telling me about a watch without actually addressing if it tells time accurately.

Spoiler: Good examples of objective reviews given in MarsAtlas post, quoted in it's entirety.

Clearly none of you have ever seen the best review site on the internet, Objective Game Reviews.

http://www.objectivegamereviews.com/

Clearly their objectivity makes them the best reviewer of games of all time as them not having any opinion makes them perfect for scoring games. I mean, just listen to this truth-bombing.

"Counter-Strike: Global Offensive is a first person shooter in which players attempt to accomplish the objective of the game mode by killing enemy players, planting bombs, or getting kills with a succession of weapons. In round-based game modes, players begin each round by purchasing equipment with money earned in previous rounds. Various weapons are available to players and the various maps have different routes the players can use to reach the objectives.


Counter-Strike: Global Offensive features eleven pistols, six shotguns, eight submachine guns, eleven assault rifles, six sniper rifles, two machine guns, a knife, six kinds of grenades, six pieces of equipment, four game modes, fourteen standard maps in competitive matches, and an integrated matchmaking system, tournament viewing system, statistics tracking system with leaderboards, and inventory system that allows the player to collect weapon skins and other items that result in cosmetic alterations and no gameplay alterations."

http://www.objectivegamereviews.com/counterstrike-go-review/

Clearly this is the best review of CS:GO ever, and there's no need for pesky "opinions" now that there's a perfect demonstartion of what the game is.

/massivesarcasm

As always, Jim Sterling comes in to save the day.


Its stupid. Penalizing a reviewer for giving an opinion is one of the silliest things I've ever heard. You don't have to agree with their opinion. There's tons of reviewers I follow whose opinions don't align with mine. I don't ***** their reviews being bad because they had different feelings about the game than I did, I recnogize that we're different human beings, and have different perspectives and feelings, and I learn more about their preferences and how they relate to my preferences. I've followed Angry Joe since 2008, and there's plenty of games that I've had a different opinion over. For example, I'm just generally not a big fan of sandbox games, so I would've personally rated games he scored highly like GTA V and Red Dead Redemption much lower than he did. Neither are in the wrong, its just personal opinion. Similarly, if a game creates a hostile environment towards the player for any reason, the reviewer should report that. Maybe you're not uncomfortable with, say, the torture scene in GTA V or Far Cry 3. Maybe they thought the scenes were just in poor taste. Maybe the reviewer thought that they were very effective for getting a point across. Maybe it was yawn-worthy and it didn't effect them in the slightest. None of those interpretations are wrong because its about their opinion, and they shouldn't be trying to posit an opinion about the game that they don't truly have, because when you're doing that, you're failing to give a genuine, honest review in favour of trying to look impartial, which is just impossible anyways.

If you want an objective game review, put a banana in front of a keyboard and see what happens.

The purpose of a review of a piece of entertainment is to tell you if it was fun for the reviewer or not, and why.

Be it a game or a stand-up comedy show.
Because even if the production values are sky high, the delivery spot on, the lighting is flawless, and the comedian almost makes you jizz yourself by merely existing (through being extremely attractive in every way), that doesn't mean the jokes can't still be shit and the actual entertainment derived from it non existent. So despite being a technical masterpiece, it can still be an extremely unsatisfying experience. An objective review would simply describe the parts and offer no judgement.

A review is a buyers guide. Not for a specific buyer, with or without certain preferences, but a buyer with the same perspective as the person doing the review. A good review will explain why the experience was fun or not, for the reviewer.

If you don't share the same perspective as the reviewer your opinion will most likely be different. That is why you should consider several reviews, and see if the same issues come up between them. This is why wildly differing review scores most likely means a game might be great to some and really bad to others, which is perfectly fine. Everyone can't think the same about everything, especially entertainment - a subjective experience.

From what I can see this is what people get upset over and call biased reviews. Reviews that disagree with their own perspective and preferences. Opinions that they disagree with and don't think should matter. Rather than reserving such allegations and complaints to reviews where the reviewer clearly has a vested interest in the success of the game or there is solid proof the review has been bought.

Let's go for an extreme example that should be horribly offensive to most people, and not in the least as interpretative as the stand-up comedy one.

Spoiler: This is, at least to me, a description of a horribly offensive and still technically excellent game. A technical masterpiece you think is so bad that shit is ashamed to be associated with it.

A game is released with excellent, stunningly high definition graphics, excellent sound design, extremely responsive controls, excellently voice acted and motion captured action, while it features dozens of the coolest most mind blowing and impressive action setpieces ever conceived by man. And the game is so well optimized and cleverly coded that it will run at 120 fps and 1080p with every possible graphical option set to maximum on a medium range PC, with ease. And you never encountered a single bug. It's a technical masterpiece of action gameplay.

However, the game is about a woman (a mother to be) unraveling the jewish and gay conspiracy secretly governing the world and threatening her family and unborn daughter. She proceeds to destroy it through a sequence of impossibly awesome action sequences that ends with raping the gay conspiracy leader straight. It sickens and offends you in every way (at least, it would for me).

Simply put; you hate the story. But other people just think the story is a context for all the awesome gameplay and don't really care that you're offended. They just want to know if the gameplay is good.

By objective standards, free of any political agendas or personal opinions or interpretations, this game is a masterpiece. But by subjective standards it is a constant stream of hate, lies and bullshit that makes you physically ill and you've never been so angry at a game in your whole life. You hate everything about it, even if it's technically brilliant and has excellent gameplay. Burning your hand would have been more enjoyable to you, as the scars would only be on the outside (and not on your very soul).

Subjectively it's a minimum score. Hell, you don't even want to grace this game with an actual review score. If you could you'd give it a negative score, because it's just off the scale horrible to you. But from an objective standpoint, or just a person who's not the least bit offended by the message of the game - they just enjoy it for the excellent gameplay - it's the best action game ever made.

Any reasonable reviewer would then, in my opinion, state that the game is indeed a technical masterpiece with excellent gameplay. And then spend most of the review explaining why they still think it's a bad game you should not buy or play. That is what a good review would mean for me.

Knowing why the reviewer didn't like this technical masterpiece I could then take this review, and compare it to that of others, in order help form form my own opinion on the game.

The subjective opinion here; "what did I think of the game and how fun was it" is what is valuable in the review. Not a description of how well the games parts work (which is also a highly subjective thing).

 

[Bellvedere]

I don't see what the issue with un/biased reviews is.

People just write about how they feel about a game. As the audience for a review, you can determine whether the aspects that they're praising or criticizing (whether socio-political or not) are things you actually care about. If you know you don't care about them then you can safely ignore it not let that affect your opinion of the game.

If you find a reviewer is constantly at odds with your thoughts on a game or focuses most of their discussion of aspects of the game you find irrelevant, then that person is probably not the best person for you to be getting reviews from, regardless of whether the reviewer is bias, corrupt or whatever. These people keep their jobs because people read their articles, so either they're crappy at their job and they'll lose it, or they're appealing to an audience that does not include you (just like perhaps the game they're reviewing might appeal to an audience that does not include them).

That's not to say there's anything wrong with criticising a reviewers opinion (in a reasonable manner), but to suggest that this is actually an issue or a problem with the games journalism industry just seems bizarre to me.

 

[veloper]

Wow, this got a lot more people talking than I expected - quite the read so far.

I suppose it's time I inject my own actual opinions on the subject at his point.

What I would be interested to know is: having read our responses, how you yourself would now answer the central question in your first post:

If I play a game where all the mechanics are excellent, production values are good, it's well optimized and the story holds up well, but I hate one aspect of the game so much it sours the whole experience (making me strongly dislike the game). How then should I then rate it?

Do you slam the game, do you merely subtract a point or just scrap the entire review?

 

[g7g7g7g7]

I did some reviewing recently and hated it, turned me off of games and writing at the same time. Doing 2 reviews to a professional websites style guide made me hate mankind. It severely limited the type of criticism I could give by censoring any mention of a person and their work on a particular project, which is madness. Then in the interest of creating a professional image style guide demanded that no use of the first person perspective to describe actions in the gameworld, meaning that actual experiences that happened to me and the things that I felt whilst playing had to be placed into the third person so what "I found [x] was [y]" would become the actions of a hypothetical third person in this example "Eagle-eyed players might find [x] to be [y]" which is fine just once or twice but by the end of the article I had invented about 10 different gaming characters to fill my shoes and it just seemed silly.

The major problem I had was their scoring system, it was literally a checklist of things, I wasn't rating a product or my experience with it, I was marking an exam paper, how much of making a game has this company achieved/10.

I have actually designed a review system that I think is balanced and fair, if there is demand for an example I will do one and if people like it I might start writing reviews again.

 

[CannibalCorpses]

What can i say? Most reviewers lack my experience of gaming so why would i even care what their opinion of a game is? My friends give me far better reviews because they know what grabs my attention, what aspects of games i like and what things break the mold for me. I see reviews as advertising (something i despise) and treat them as such...