106: To Hear Ourselves Review

[Russ Pitts]

To Hear Ourselves Review

"From the developer's point of view, reviews are simple: They're either good or bad. If the review is good, chances are sales will be good, too (exception: Psychonauts). If the review is bad, all is not quite lost, as sometimes even a bad review will move copies (see: most EA games), but it's usually bad news. So, as far as a developer is concerned, there's a lot riding on whether or not the person his PR flak sends a review copy to likes it. The frustrating part - for developers - is that no matter how much effort they put into a game, no matter how perfectly they polish it, there's no guarantee they'll receive a good review - or any review at all."

Russ Pitts reviews the current state of the art of game reviews.

Read Full Article

 

[darshannon]

In an age when nearly anyone can start a website and write about games, and most every type of gamer can find at least one popular message board (full of other people who are also playing games, and writing about them) to suit his needs, the need for formal reviews would seem to be diminishing. Or at least the need to trust the opinions of people you don't know.

The thing is - some people think (although they would never admit it) that magazines are made not by mere mortals, but by some otherworldly, somewhat superior beings. Just like movies or TV shows. And that's why they need magazine reviews. Because, when one of their friends says that game X sucks or is a greatest game ever, it's one thing. But when it is written in a magazine, it's something completely different. For them anyway.

 

[Dr. Wiley]

I was particularly interested in the idea of game reviewer as someone who plays dozens of games a year to completion. Certainly, a good reviewer will be well versed in all the most recent games, but I don't agree that familiarity necessarily breeds good instincts. For me, the best reviewer is one that situates a game in the ongoing narrative of gaming and picks apart the games relevance, if it does indeed have relevance.

Earlier this year there was a lot of back and forth between people in the movie industry about the value of critics. It was interesting that a lot of the same issues that drove their debate are mentioned here with regards to gaming. What came out of that debate, in my opinion, and to a certain extent comes out of this piece, is the idea that many times the reviewers are too close to the subject to make an a fair judgement on a game's quality. Book review editors tend to work around this problem by assigning reviews to writers who don't necessarily function as book reviewers full time. This might be a bad example, because any writer worth his or her salt obviously reads many books to completion every year, and thus could be said to have a strong connection to the world of books, but I would argue that even today, game reviews need not be handled by a specialized class of writers who focus on reviews. Instead, culturally aware and gaming literate writers can and do review games, and often bring fresh, interesting perspectives that might not otherwise be heard.

Then again though, as Mr. Pitts mentions, we run into the problem of the wants and needs of the gaming community who by and large want to hear very specific (one might even say formulaic) things about any particular game that is being reviewed. As we heard from developers, reviews do serve the purpose of showing how and why a game might be better, but if game reviews continue to operate within the semi-closed community of harcore-to-moderate gamers they will falter at serving a larger purpose of situating games and gaming within a larger cultural sphere. Maybe, a majority of readers don't want that to happen, but I think that it would be a good goal to strive for.

 

[Virgil]

Huh? Don't you mean "it isn't unusual either"? I'm not sure what impression I'm getting, here.

The article presents an anecdote and states that while the situation isn't a unique case, events like it are still an exception to the norm (that is, unusual). This is reiterated a little further in the article:

Still, altering a review to avoid negative vibes is typically frowned upon, no matter whose game it is. The first question Arthur's usually asked after recounting his story is, "Why didn't this guy get fired?"

 

[Russ Pitts]

Wiley, what you're describing sounds a lot more like criticism than review. There is a difference, and it lies exactly along the line of overarching cultural relevance you suggest. We don't currently have a lot of critics in gaming.

 

[Russ Pitts]

I'm not sure, Brother. And I'm not in any way suggesting change is exactly necessary, even. What I think is most interesting about the state of reviews is that, in spite of how much thought, consternation and analysis that goes into them from all parties concerned, most people just look at the numbers.

Now does this mean that the growing criticism movement will be largely ignored? I don't know. But I recall someone warning us about the kind of content we put together at The Escapist a long time ago, and we seem to have done quite well for ourselves.

Every person I interviewed for this piece (even those who didn't make it in time for the article) believes the system needs to change, but I'm not so sure. Further, I'm not even sure it _could_ change, no matter how much we wanted it to.

I think the nature of game reviews mirrors the nature of the gamer market: we're all largely internet literate and therefore used to seeking out reviews and iopini0ons online. Which, in another turn, is where we go to express our opinions of games when we have them. It's a system that's evolved to fit this peculiar market, and even if it worked perfectly, even if there were no Arthurs and no Tom Chick reviews of Deus Ex, there'd still be so many people involved, so many lives in the balance, that friction would be inevitable.

As for blogs vs. traditional media, I don't really see a competition there. I see an evolution. The blogs and amateur sites are experimenting with new media formats, and pioneering a lot of really interesting ways of interacting with the audience. This is as it should be. Big corporations can't afford to take a lot of risks, but when your bottom line equals your own personal disposable cash, you can do whatever you want.

What;'s happening now, is you're seeing established media sites adopting and assimilating stuff that's been proving across multiple amateur sites and adopting their message to fit the audience that has grown up around those formats.

But now we're on another topic altogether ...

 

[Dom Camus]

I think it's pretty sad that Flow review came in for criticism, because that's exactly the kind of review I want to encourage. From my perspective Flow's one of the most desperately overrated titles in the history of gaming. Sure, it's got things in its favour, but at the point at which that review was written the main thing which needed saying about the game is that it didn't deserve 1/10 of the column inches wasted on it. It's an unremarkable casual game which would have done nothing more than vanish harmlessly without a trace if it had simply appeared on a few casual portals.

 

[turkmenistan]

People often compare game crit to rock crit. But the first rock critics - XGau, Metzler - were intelligent, aggressive and creative writers who understood criticism (old-school, literary-style), hated authority and wrote great pieces. Some people want more gonzo critics, eg Bangs, but I'd kill to have a Robert Christgau of gaming - someone with an encyclopedic knowledge who could still get to the real point in 100 words.

We can talk about systematic problems as much as we want but the real issue is that none of the writers have stepped up to the plate, and none of their editors are challenging them. Every newspaper in the country has to run game reviews, but they hire their 13-year-old nephew to write them up for other 13-year-olds. Even that "Game Culture" piece in the New York Times is an abortion (and generally, the NYT knows how to do tech). Gamespot is knowledgable but the writing's dreary. Bloggers are idiots.

The system will change when one or two great writers step up and change it. The bar will finally be set higher, and people will have a reason to ask for more than just a rating and some bitching about the graphics.

 

[Ajar]

Thanks for mentioning that New Yorker piece, I just read it and it was very enjoyable.

 

[FunkyJ]

I've got a story to relate.

A certain company sent our little website one of their big name games. One which was delayed coming out to the console we review for. One that, despite being good, was not as good as it's predecessor.

Our writer reviewed it, gave it 90 out of 100. Now, that's a good score, don't you think?

Well, the company didn't appreciate the 90/100, and called the writer a hack, and threatened the editor of our site with a black ban if the score wasn't increased.

The editor basically told Rocks... er I mean the company... to go F*ck themselves.

Now, when a little fan-run website gets treated like that, I can only imagine they type of crap bigger websites who rely on advertising dollars and such have to go through

 

[mactbone [deprecated]]

I'm just curious why gaming reviews have to be treated in such a different manner from movie and book reviews. Part of it is probably that there are book and movie reviews that have good enough writing to be considered literature. I've never had anyone tell me that I needed to check out a game review because it had great writing. A review isn't just about telling people good or bad it's a form of art in itself and has been for a long time.

 

[Troy Goodfellow]

Now, when a little fan-run website gets treated like that, I can only imagine they type of crap bigger websites who rely on advertising dollars and such have to go through

The great irony is that the bigger websites who rely on advertising dollars are, in fact, more independent of this kind of pressure since the game companies can't blacklist or cut them off as easily. If your game isn't reviewed by Gamespot, people notice.

The only proper response to that kind of pressure is "Can I quote you on that?"

 

[Julian Murdoch]

I'm sorry, if you're relationship with a publisher is predicated on being able to get free games, it's nearly impossible that anyone is going to be taking your opinions that seriously in the first place. I'm not sure which kinds of fan-run websites we're talking about here, but I know that most of the ones I read are the ones where there is actual content written by people who at least try and write well, paid or not. And in almost none of those cases do I get the sense that a game is being written about either because of, or in response to, the delivery of a free copy.

I'm not being all high and mighty here. I don't immediately delete any preview code I get from PR. I don't return-to-sender every Sharpie-lettered DVD I get in the mail. My point, if I have one, is that these things end up self-correcting in the long run. Sure, there are unscrupulous inside baseball shenanigans that happen inside entertainment publications. Sorry if I'm not shocked. But I believe that better writing, consistency of editorial policy and treating readers with respect eventually win out over time.

And I don't think any of that has anything to do with the whole "NGJ" rat hole. Calling something by a different name has zero impact on the quality of the actual work. If you want it better or different, just write it better or different. If there's no market for it, then either don't worry about trying to make a living at it, or figure out how to convince your editors to publish it the way you want to write it. And if you can't do either of those things, well, maybe there's a lesson in that.

Journalism, even deeply intellectual and brilliantly crafted, is fundamentally commercial writing. You get paid by the word, the gig, or the hour. You do the very best work you can do and still get the client to pay for it. If they don't like it, they don't pay you, and guess what, they win. CNet/ZD/Future/Bob's Discount Gaming Blog -- they're just a different kind of client. As a writer, you have the option to either work, or not work.

Maybe I'm a tired old curmudgeonly writer who's lost all my love for life, but I don't think so. I just like to eat and pay my kids' tuition bills.

The best two sentences of this whole issue-long and beaten-to-death debate were off-hand in this thread:

"Every person I interviewed for this piece (even those who didn't make it in time for the article) believes the system needs to change, but I'm not so sure. Further, I'm not even sure it _could_ change, no matter how much we wanted it to."

 

[Troy Goodfellow]

I'm sorry, if you're relationship with a publisher is predicated on being able to get free games, it's nearly impossible that anyone is going to be taking your opinions that seriously in the first place. I'm not sure which kinds of fan-run websites we're talking about here, but I know that most of the ones I read are the ones where there is actual content written by people who at least try and write well, paid or not. And in almost none of those cases do I get the sense that a game is being written about either because of, or in response to, the delivery of a free copy.

If this was directed at me, I didn't mean to imply that fan run websites necessarily corrupt. But since many of them do get free review copies, their work is as susceptible to scrutiny as the big boys, but without the fallback of being "important enough" to not push around. Mind you, smaller sites are less likely to have personal relationships with developers or PR and the number of writers moving to development further complicates coverage. I guess, in the end, honest people will be honest and dishonest people won't.

Readers will think everyone's corrupt in any case; they think this of journalism in general. Professionalism should apply even if you're an amateur, but the internet is not exactly a place where everyone assumes the best of people.

 

[Cordelia]

Readers will think everyone's corrupt in any case; they think this of journalism in general. Professionalism should apply even if you're an amateur, but the internet is not exactly a place where everyone assumes the best of people.

So very true. More to the point, they tend to base their analysis of your corruption on how closely your opinion aligns with theirs. If you agree with them, then of course you're as honest as can be, but if you disagree, you're a liar/shill/on the take, etc.

I think the review process would improve if the readers were made more aware of how a particular outfit conducted theirs. There was one site I used to read, for example, that said up front that they only played through the first 4-8 hours or so of a game, so that their reviews were more "first impressions" than in-depth analyses. Their reasoning was that if a game didn't grab you in that amount of time, it was unlikely that you were going to stick around for the ZOMG at hour 33, so why should they bother playing it that long? Whether or not you agree with that particular sentiment, it at least gave their reviews some context.

 

[FunkyJ]

Readers will think everyone's corrupt in any case; they think this of journalism in general. Professionalism should apply even if you're an amateur, but the internet is not exactly a place where everyone assumes the best of people.

Problem is, professionalism in journalism isn't exactly rampant these days. Paying for stories, embedded journalism, PR leaks from government departments, PR/AP feed re-writes, no fact checking, editorial and advertorial interference, adverts posing as editorial, plagiarism, constrictive media and asset ownership...

I know that poor circulation with newspaper means budgets are being slashed, international bureaus being closed, yadda yadda but really there's no excuse for some of the mistakes that happen.

I've always felt that calling writing about games "games journalism" does it a huge disservice.

Kinda related to this is an interview with the people behind the biggest games blogs: http://kotaku.com/gaming/clips/kotaku-unlocks-the-bonus-round-part-i-281131.php

Interesting to see how they view themselves...

 

[juicy]

"Game reviewers want every game to be Zelda," says Game Daily's anonymous game journalism critic, Mr. Media Coverage. "That's what one developer told me. He said that the reality of game development is that most developers make games for a very specific target audience, and the developers do their best to find and meet the needs of those specific gamers. It's a frustration, then, when game reviewers complain that the game is too 'kiddie' or too 'redneck' or too targeted to one group. That, after all, was the entire purpose of the game."

So true. I work in development, and we are usually given very specific directions as to who our audience is (market), genre, etc. I read reviews and am often questioning the integrity of the game sites 'critics' who mostly seem biased towards AAA first person shooters, rather than thinking globally and objectively about the game and its target market.

Games today aren't just for 14 year old pimply boys. MOST games today are made for the mass market - for casual players and their families. When a great kids game is made, however, suitable for this market, how often does it receive a high score? Rarely. Because it's not considered AAA, or the reviewer can't see beyond his/her own nose.

Yes, a new voice for game reviews is needed. A diplomatic one that can see a game through the eyes of the user the game was intended for, and one who isn't swayed by bullying publishers.

I also think the rating system really does need a good shake up.

"Most members of the games press are people who enjoy telling people about good games they got to play. Hopefully your game will be one of them." and.. "...I'd like to see reviewers express a more consistent ... philosophical approach to games, so readers could find the [reviewers] they trust and stick with them a little more."

I wonder how many members of the games press have actually worked in the games industry? Maybe if they did they'd be better at making more educated judgements.