Topical Tuesday: Does a Ranking System Still Carry Weight?
- Thread starter Spinwhiz
- Start date
I remember a time when a ranking system meant something to me. It was the 90s and I wasn't even a teenager yet. I would read my monthly gaming magazine and await with bated breath the percentage score that would be given to whatever game I had been keeping my eye on. Some spectacular image from the game would beckon from the front cover, and I'd dive in, eagerly seeking out the results.
My preteen gaming life was defined by that magazine. If a game got more than 80%, I'd save up my allowance and buy it. If not, I wouldn't. I had utter faith that these wise journalists would never steer me wrong. And you know, they never did. I kept buying that magazine because my trust was always rewarded.
(I also loved the yearly "Top 100 Games of All Time" lists. I prided myself on how many of the top 15 or so I'd played at any given time, and learned about some great oldies in the process.)
Something is different now. I don't buy magazines any more, and it seems that either the standard of the journalism is worse, or I've changed in my approach to games. Probably both.
For one, my taste in games is more defined. I don't mean it's better, just that I know what I like. Once a gamer works that out, I think, score-based reviews automatically mean less; I can look at a game that wins 9/10s everywhere on the net and know at a glance that I'm still not going to like it, while I can see a 7/10 in my favourite genre and consider giving it a go, though I never would have all those years ago.
I suspect that the journalistic standard has been cheapened by the ubiquity of online gaming sites, too. When one in ten gaming fanatics believes he knows better than the last reviewer he read, and there's room for said fanatics to voice their opinions in a professional-looking medium, it can hardly be surprising if there are a lot of unwarranted egos in the mix. Add to that a lack of any journalistic approach and you have a situation in which finding valuable ideas is like panning for gold.
I would say that the best thing for serious gamers to do would be to simply find out about the games they are interested in through discussion with other gamers - just as they already do. Score-based reviews might still have a place for new or casual gamers, but if I were in that position again, I'd be seeking out magazines, just like I did as a kid. Assuming they're still as entertaining and trustworthy as they used to be, of course.
My preteen gaming life was defined by that magazine. If a game got more than 80%, I'd save up my allowance and buy it. If not, I wouldn't. I had utter faith that these wise journalists would never steer me wrong. And you know, they never did. I kept buying that magazine because my trust was always rewarded.
(I also loved the yearly "Top 100 Games of All Time" lists. I prided myself on how many of the top 15 or so I'd played at any given time, and learned about some great oldies in the process.)
Something is different now. I don't buy magazines any more, and it seems that either the standard of the journalism is worse, or I've changed in my approach to games. Probably both.
For one, my taste in games is more defined. I don't mean it's better, just that I know what I like. Once a gamer works that out, I think, score-based reviews automatically mean less; I can look at a game that wins 9/10s everywhere on the net and know at a glance that I'm still not going to like it, while I can see a 7/10 in my favourite genre and consider giving it a go, though I never would have all those years ago.
I suspect that the journalistic standard has been cheapened by the ubiquity of online gaming sites, too. When one in ten gaming fanatics believes he knows better than the last reviewer he read, and there's room for said fanatics to voice their opinions in a professional-looking medium, it can hardly be surprising if there are a lot of unwarranted egos in the mix. Add to that a lack of any journalistic approach and you have a situation in which finding valuable ideas is like panning for gold.
I would say that the best thing for serious gamers to do would be to simply find out about the games they are interested in through discussion with other gamers - just as they already do. Score-based reviews might still have a place for new or casual gamers, but if I were in that position again, I'd be seeking out magazines, just like I did as a kid. Assuming they're still as entertaining and trustworthy as they used to be, of course.
Ranking? It's the ultimate summary of the reviwer's experience. If a reader can carry nothing else away from the review, they can take: "*suchandsuchareviewer* gave it 8.5 out to 10!"
With a reviewer you know, you read their monthly or weekly column, you get to know and understand where they are coming from and their preferences. You know they like RPG's, but dislike buggy controls. You know they automatically take away a point for bad voiceacting, cause they don't like it, but will be very generous when it comes to quicktime events, because they just don't bother that reviewer much.
With such a reviewer, then yes, a ranking makes sense and helps. You can then work out how closely your tastes dovetail with his, and then the ranking falls into place. "Ok, *suchanduchareviewer* likes RPG's, particularly Bioware ones, just like me, and for much the same reasons. This is an RPG, so the ranking is likely to be close to the one I'd give it. 8.5? Let's try it."
However, most people don't have such an experience anymore. Magazines devoted to gaming have dropped in popularity, and the rise of gaming sites, while good, mean that the multitude of voices giving their opinion of a game can drown out the one or two people whose views might actually help and guide you. So rankings lose their meaning.
In short, rankings are only as good as the credibility of the person doing the reviewing and the trust the reader places in him. And seeing as both of those criteria are being eroded in this inernet age, rankings are correspondingly falling in value.
With a reviewer you know, you read their monthly or weekly column, you get to know and understand where they are coming from and their preferences. You know they like RPG's, but dislike buggy controls. You know they automatically take away a point for bad voiceacting, cause they don't like it, but will be very generous when it comes to quicktime events, because they just don't bother that reviewer much.
With such a reviewer, then yes, a ranking makes sense and helps. You can then work out how closely your tastes dovetail with his, and then the ranking falls into place. "Ok, *suchanduchareviewer* likes RPG's, particularly Bioware ones, just like me, and for much the same reasons. This is an RPG, so the ranking is likely to be close to the one I'd give it. 8.5? Let's try it."
However, most people don't have such an experience anymore. Magazines devoted to gaming have dropped in popularity, and the rise of gaming sites, while good, mean that the multitude of voices giving their opinion of a game can drown out the one or two people whose views might actually help and guide you. So rankings lose their meaning.
In short, rankings are only as good as the credibility of the person doing the reviewing and the trust the reader places in him. And seeing as both of those criteria are being eroded in this inernet age, rankings are correspondingly falling in value.
The only scores I ever look at are Metacritic, due to it being an average of many different reviews. But more importantly, I read the reasons for it.
Game reviews are based on opinion and preferences of a reviewer and as such will never be objective or applicable for all of his followers. That's one thing, now add to that scoring under pressure of hype and your score is pretty much worthless in long term. I mean, yeah, for time being even Final Fantasy 8 was epic etc. but just after few years ppl realized that it wasn't that good. Such overrating hyped games (under menace of lynch) leaded us to where we are currently - fresh new game is given high notes, just to show that it is better than hyped game that came earlier, which wasn't all that great (f.e Halo)
I feel that the ranking system itself has become ineffective, simply because all publications have different internal criteria as to what goes into the score. Some might have graphics as a subscore, while others might not take graphics into the final score at all (or just have a checkbox in regards to passability). Others leave the score up to the whim of the reviewer himself, meaning that the final "score" on the review can either reflect the review as a whole, or smash it with a hammer ("This game sucks. 10/10"). Without a standardized format, rankings by themselves have no merit--and it's hard to standardized an opinion of what is a largely subjective experience. Until then, we're stuck with the many different review systems and their scoring systems (97%, ***/*****, 4/10, etc.) and scoring criteria.
There are other reasons, too--namely spin and buy-outs. Neither is as prevalent (hopefully), but still a contributing factor. Buy-outs should be obvious: if Company X threatens to pull advertising from Magazine Y if Product Z gets a bad score/review, then the odds are that Magazine Y will inflate the score of Product Z and give it a glowing review, even if the product was bland, broken, or otherwise deserving of a low score. Spin, on the other hand, is harder to define, but generally it's bias, both on the parts of the reviewer and the reviewed. If Reviewer Beta is a fan of Genre Delta, and plays a game from Genre Upsilon--a genre he despises--will more than likely give it a lower score, while Reviewer Phi--who loves Genre Upsilon--can either give it a better score due to familiarity with the genre (and forgive its shortcomings), or bash it completely (because it's not like Game ZZ). The flip-side to the other isn't as predominant, as it mainly only applies to games made by smaller companies (i.e., by only one person). This side is willing to take lower scores as something good, and then apply its own unique spin to it to make it seem positive. Hell, I've seen someone say that a score of 2/5 was a great score since it came from someone who didn't really like the game in the first place. And this came from a game made only by one person.
Rankings are becoming obsolete--and with time, it's likely that reviews will as well if more of them begin to succumb to corporate pressure. Hell, one can argue that reviews themselves are already obsolete, seeing as they attempt to quantify a subjective, data-less experience and, at best, merely offer a guess as to whether the consumer reading the review will like the game or not.
There are other reasons, too--namely spin and buy-outs. Neither is as prevalent (hopefully), but still a contributing factor. Buy-outs should be obvious: if Company X threatens to pull advertising from Magazine Y if Product Z gets a bad score/review, then the odds are that Magazine Y will inflate the score of Product Z and give it a glowing review, even if the product was bland, broken, or otherwise deserving of a low score. Spin, on the other hand, is harder to define, but generally it's bias, both on the parts of the reviewer and the reviewed. If Reviewer Beta is a fan of Genre Delta, and plays a game from Genre Upsilon--a genre he despises--will more than likely give it a lower score, while Reviewer Phi--who loves Genre Upsilon--can either give it a better score due to familiarity with the genre (and forgive its shortcomings), or bash it completely (because it's not like Game ZZ). The flip-side to the other isn't as predominant, as it mainly only applies to games made by smaller companies (i.e., by only one person). This side is willing to take lower scores as something good, and then apply its own unique spin to it to make it seem positive. Hell, I've seen someone say that a score of 2/5 was a great score since it came from someone who didn't really like the game in the first place. And this came from a game made only by one person.
Rankings are becoming obsolete--and with time, it's likely that reviews will as well if more of them begin to succumb to corporate pressure. Hell, one can argue that reviews themselves are already obsolete, seeing as they attempt to quantify a subjective, data-less experience and, at best, merely offer a guess as to whether the consumer reading the review will like the game or not.
Scores and rankings... for me they've always represented laziness from both the reviewer's and the readers' perspectives. Slapping a number on the end of reviews can lead many to skip over the review itself just to see what score is associated with it, without taking into account what was specifically said about the game. I know from experience that after seeing the 9s and 10s given to MGS4 I went out to purchase it, only to find later that the reviewers had talked about a lot of problems in their reviews; problems that made the game almost unplayable for me personally.
You also get occasions similar to this where the score does not reflect the review at all. You see many reviews where there are a few problems with the game, but the game was given a 10 (or a 5 or whatever the highest one is). This doesn't make sense as surely 10 represents "the perfect game", not a great game with flaws. I suppose this might link in to much of the thinly veiled bribery (in the form of endorsements, adventising etc.) that plagues many gaming websites today.
What scores are very good for is comparisons between games, but this too is a bad thing. Am I supposed to believe that Fat Princess is a much better game than Alpha Protocol? Even though it's a very basic and simplistic game with little variety, while Alpha Protocol is a good game with a few flaws but nevertheless offers a much better experience. The difference in score on Metacritic by 16 (81-65) tells me I should. Maybe Fat Princess is good for a cheap PSN game but I'm sure that it would have gotten much worse scores if it retailed at full price. A stand-alone review would tell you the facts so you could make your own decision on the game.
I would much prefer it if all reviewers dropped any type of scoring system, but since that will never happen as companies are always obsessed with numbers, at least make it a vague one. Percentage scores always tend to make me laugh as there is no way that you can be that precise with opinion.
You also get occasions similar to this where the score does not reflect the review at all. You see many reviews where there are a few problems with the game, but the game was given a 10 (or a 5 or whatever the highest one is). This doesn't make sense as surely 10 represents "the perfect game", not a great game with flaws. I suppose this might link in to much of the thinly veiled bribery (in the form of endorsements, adventising etc.) that plagues many gaming websites today.
What scores are very good for is comparisons between games, but this too is a bad thing. Am I supposed to believe that Fat Princess is a much better game than Alpha Protocol? Even though it's a very basic and simplistic game with little variety, while Alpha Protocol is a good game with a few flaws but nevertheless offers a much better experience. The difference in score on Metacritic by 16 (81-65) tells me I should. Maybe Fat Princess is good for a cheap PSN game but I'm sure that it would have gotten much worse scores if it retailed at full price. A stand-alone review would tell you the facts so you could make your own decision on the game.
I would much prefer it if all reviewers dropped any type of scoring system, but since that will never happen as companies are always obsessed with numbers, at least make it a vague one. Percentage scores always tend to make me laugh as there is no way that you can be that precise with opinion.
Reviews help, but I honestly read many reviews for entertainment. These days I can get a pretty good sense of whether a game is going to be quality or not and whether I'm going to enjoy it or not through the myriad cinematic trailers, gameplay trailers, developer diaries, and downloadable demos the internet has to offer. I'll often know whether or not I'm going to get a game long before the review comes out.
Though sometimes a really positive review (or rather, a number of positive reviews) will compel me to take a second look at a game I brushed aside; likewise, negative reviews will make me think twice before making a day one purchase. But in the end, I try best to think for myself.
Though sometimes a really positive review (or rather, a number of positive reviews) will compel me to take a second look at a game I brushed aside; likewise, negative reviews will make me think twice before making a day one purchase. But in the end, I try best to think for myself.
For me, reviews are just a stepping stone to understanding if you like a game. Every gamer has their own likes and dislikes, and what for one person could be a game-breaking problem could for one person be a minor annoyance. It just depends. Personally, I'm not a huge fan of sports games. If I see the newest Madden game get impressive scores from tons of different places, am I suddenly going to want to run to the nearest store and pick it up? Of course not.
But, for example, let's look at a more recent game, Alpha Protocol. A lot of different places I looked gave the game low scores, tons of scores ranging from 3-6. Reading the reviews, most places mentioned that the game had tons of potential and that your choices really did make huge differences to the game. They also mentioned that the game was marred with bugs and poor controls. Me, I can live with bugs, and I can work past bad controls. I personally love feeling like I affect the game that I'm playing, and I enjoy games that give me large amounts of control. I picked up the game, and enjoyed it all the way through. Were there bugs? Yes. Were the controls pretty bad? Yep. But because I enjoyed the game for the potential it showed and the control over the story, I was able to look past the problems and have a great gaming experience.
Mostly, my point is that a review should never make or break your decision to buy a game. It's just somebody else's opinion. Sure, they may be *paid* for their opinion, but it's just an opinion none the less. When I buy games, I prefer to go to boards and such places and listen to the community. See what they love about the games. Despite the fact that reviewers across the board despise them, I am a HUGE fan of the Dynasty Warriors games. I just love being able to mindlessly kill thousands of people. I find that to be a good way to spend an afternoon.
But yeah, the ranking system is the curse of all reviews. You see a 6 on a game and you run away, and a 9 on a game and you run toward it with open arms, but they're just a numerical value attached to an opinion. What matters is the opinion itself. Who cares that "Bloody-Blood Kill Die 6: The Death of the Infants" is being given 10's from every reviewer ever made ever, if you can't appreciate the subject material, than you aren't even going to give it a second look (except maybe for the name, because that's just ballsy.)
Final Summation: Ratings can be important if you take them with a grain of salt, but Numerical Rankings are as obsolete now as they were when they were invented. They may be an easy way to see how "good" a game is or not, but they aren't helpful enough to let you skip the review. And if you need to review anyway, what's the point of the rating? But they'll never vanish, because the ADHD crowd would scream and shout their way to hell and back that they can't understand the quality of a game with a glance.
But, for example, let's look at a more recent game, Alpha Protocol. A lot of different places I looked gave the game low scores, tons of scores ranging from 3-6. Reading the reviews, most places mentioned that the game had tons of potential and that your choices really did make huge differences to the game. They also mentioned that the game was marred with bugs and poor controls. Me, I can live with bugs, and I can work past bad controls. I personally love feeling like I affect the game that I'm playing, and I enjoy games that give me large amounts of control. I picked up the game, and enjoyed it all the way through. Were there bugs? Yes. Were the controls pretty bad? Yep. But because I enjoyed the game for the potential it showed and the control over the story, I was able to look past the problems and have a great gaming experience.
Mostly, my point is that a review should never make or break your decision to buy a game. It's just somebody else's opinion. Sure, they may be *paid* for their opinion, but it's just an opinion none the less. When I buy games, I prefer to go to boards and such places and listen to the community. See what they love about the games. Despite the fact that reviewers across the board despise them, I am a HUGE fan of the Dynasty Warriors games. I just love being able to mindlessly kill thousands of people. I find that to be a good way to spend an afternoon.
But yeah, the ranking system is the curse of all reviews. You see a 6 on a game and you run away, and a 9 on a game and you run toward it with open arms, but they're just a numerical value attached to an opinion. What matters is the opinion itself. Who cares that "Bloody-Blood Kill Die 6: The Death of the Infants" is being given 10's from every reviewer ever made ever, if you can't appreciate the subject material, than you aren't even going to give it a second look (except maybe for the name, because that's just ballsy.)
Final Summation: Ratings can be important if you take them with a grain of salt, but Numerical Rankings are as obsolete now as they were when they were invented. They may be an easy way to see how "good" a game is or not, but they aren't helpful enough to let you skip the review. And if you need to review anyway, what's the point of the rating? But they'll never vanish, because the ADHD crowd would scream and shout their way to hell and back that they can't understand the quality of a game with a glance.
For me, the more information I am provided about something, the better. So I enjoy reading multiple reviews of games and seeing multiple Rankings/Scores for games. I don't base my buying decision on reading just 1 review or seeing 1 score for a game, so the bias of 1 or 2 reviewers is negated by this.
Because of this I think Rankings are still perfectly valid to use and if nothing else help as a reminder about games long after you have forgotten what the review actually said.
A review needs 'closure', it needs a conclusion and it needs to stick a flag in the ground to show what the reviewer has decided. And, for me, the only way to comprehensively do that is by Ranking the game.
Because of this I think Rankings are still perfectly valid to use and if nothing else help as a reminder about games long after you have forgotten what the review actually said.
A review needs 'closure', it needs a conclusion and it needs to stick a flag in the ground to show what the reviewer has decided. And, for me, the only way to comprehensively do that is by Ranking the game.