Separator

Metacritic isn’t the problem. Gamers are.

Rm_headshot
Thursday, November 17, 2011

Here's an idea. Let’s have a calm, rational discussion about video-game reviews.

“I haven't played Zelda yet but F***k you GameSpot!”

- Raymus79

We can do that, right? Sit down like rational adults and hash a few things out? Because it’s time we got a few things straight on the subjects of opinion, critical thinking, expectations, and responsibility.

“I dont like to criticise [sic] a review without playing the game first but this one is off completely and you only have to think rationally to get to that conclusion.”

- mazongo

Zelda: Skyward Sword
Eff this kid's toy. I want a shotty.

I’ll start with the most obvious thing imaginable. If you do not speak from a place of first-hand knowledge, then representing a baseless opinion -- or worse, a Metacritic number -- as an iron-clad fact is the act of a moron. And every time I see it happen, one thought pops into my head.

“Fire this retard”

- D3dr0_0

 

What brings this to mind? GameSpot reviewer Tom McShea hit The Legend of Zelda: Skyward Sword with a 7.5 review score, way below its Metacritic aggregate of 94. It’s fair to call McShea’s score an outlier and more than fair to disagree with the points he raises...though personally, I can’t. I haven’t played more than a demo of Skyward Sword yet. And seeing how the game won’t release to the public for another few days, neither has anybody currently demanding McShea’s head on a spike.

Least you think I’m singling out one breed of fanatic, gamers also screamed bloody murder at any reviewer who dared score Uncharted 3: Drake's Deception less than a perfect 10, while Destructiod’s Jim Sterling induced well over his weekly quota of nerd rage by handing Gears of War 3 an 8.0 (though in fairness, his review read more like a 9). And that's just in the last month.

"The fact that big name reviewers like IGN and Game Informer gave this a rare 10 score, and also the fact that this 7.5 score is a full 2 points lower than the Metacritic average, I have to say I'm really finding it hard to trust this review."

- Superzone

That guy sounds far more reasonable than most, but look closer. He’s actually using opinions he likes in order to “disprove” an opinion he doesn’t like without ever having an opinion of his own. And I'm being overly generous when I say "opinion," because what he's really using are numbers. If he'd said "Richard George at IGN and Phil Kollar at Game Informer, two reviewers whose opinions I respect, gave Skyward Sword perfect scores for these reasons," I'd applaud him. As-is, I use him as an example of lazy non-thinking.

See, Metacritic basically takes a bunch of numbers and averages them. That's useful as a general barometer, but it doesn't tell you anything specific...and it tells you nothing whatsoever about the game itself. Not what it did right. Not what it did wrong. Not whether you might enjoy playing it.

Zelda Skyward Sword
But where are the beer steins and sausages?

Metacritic itself isn't the problem, but people who slavishly believe an aggregate score -- or indeed, any numerical value absent of context -- are. If you make decisions purely off a metascore, you've essentially surrendered your ability to make decisions to 30-odd strangers who don't agree with each other. Basing an opinion off a metascore means you don't have an opinion at all. You have a number determined by a calculator, and you don't even know why it's that particular number. Congratulations.

And yet, a lot people reference Metacritic as the gold standard of critical thought when it's really nothing more than simple arithmetic.

"He can have his opinion, but I don't want an opinion, I want someone to tell me the quality of something, no matter what he likes or not. And 7.5 is NOT the quality of this game."

- Alosjs

Ah, now, you see? There. Right there. That’s a guy who’s not fulfilling his responsibilities as a sentient creature. He wants someone else to do his thinking for him but only if they tell him what he wants to hear. When that didn't happen, he went ahead and provided his own (admittedly short and vague) review despite no hands-on time with the game. Well, Alosjs, if it makes you feel better, I'm a professional game reviewer, and even though I haven't touched Skyward Sword either, I hereby give it an official Bitmob review score of 9.0. Go ahead and add that to Metacritic.

While you're at it, ask how you'd feel if professional sites really did score games without playing them first the way a lot of commenters do.

"What? Nintendo didn't want to pay you for a good review, Gamespot?"

- Death_Blade_182

Hey, I get it. Gamers want games to be awesome. So do I. But it's our responsibility as reviewers and critics to make sure you know what you're buying into and leave you a better-informed consumer. If we have concerns, it's our job to relate them...and it's your job to locate reviewers you trust and evaluate everything they tell you, good and bad. You must step up, commit to hearing everything the actual review says, and carefully scrutinize a perfect 10 at least as much as a 7.5 even if that's what you hoped for. I'm not saying we're super-geniuses and you should hang on our every golden word, but we are in a position to get information before you do, play the games before you do, and even speak directly to the people who make them. We are very well informed. A savvy consumer takes advantage of that. A smart consumer applies a little critical evaluation of their own to the task.

Zelda Skyward Sword
Red man has blue tongue. Blue plant has red tongue. This must be a puzzle!

When I read a review, I always check who wrote it, first thing. Some people I trust and some I don't, and I know all their names. I seek out the former and avoid the latter, and even then, I read every word carefully to seperate out their tastes from mine. Maybe Tom McShea doesn't reflect how and why you game, so put him on your do-not-follow list...but do yourself a favor. Play Skyward Sword first, and see if maybe he did make an important point or two.

If, on the other hand, you’re simply looking for someone to reinforce a decision you already made, stick to the fanboy press. HipHopGamer will love all the extra attention.

I'm going to suggest a different course. Don't give a brand a free pass just because it's a brand you like. Don't assume greater knowledge before the game's physically in your hands. Stop obsessing over numbers and start obsessing over words and ideas. Above all, measure your expectations, and set your standards higher. Demand more scrutiny, more nuance, less gushing praise, and more honesty, however brutal it gets. Great games have flaws. Flawed games can have sparks of greatness. Nothing's perfect, least of all people.

"7.5 ??? what a joke, this is must be the crappiest unprofessional review i have ever seen, all sites gave it 10/10. your clearly a fail. all your dislikes points proves that your a big NOOB."

- 64-bit

But we can sure as hell be better than this.

 
Problem? Report this post
RUS MCLAUGHLIN'S SPONSOR
Comments (21)
Default_picture
November 17, 2011

I've never understood the venom directed at Metacritic. The site is no more or less than a compilation of critics' reviews and users' less-than-informed opinions. As such, I use it all the time as a resource in the same way I use Wikipedia as a launching point to seek out aggregated source materials. And I ignore the chatter from the peanut gallery.

Pict0079-web
November 17, 2011

I think everyone first questioned the site's credibility when Sega began taking tons of Sonic games off the shelves, just because their Metascore was bad. That was when everyone kept wondering why developers paid more attention to the score, rather than the factors that reviewers were basing their judgement on.

These days though, I think it's more or less an accepted fact that a Metascore isn't the final word. I appreciate that Metacritic isn't blatantly proclaiming anything as the absolute best anymore. Instead, they just put the scores for the week in a feature article. It makes a review less like a hard judgement and more like a simple aggregate.

Besides, I still think Kanye West is over-rated on that website.

Default_picture
November 18, 2011

Right, the aggregate score is misleading, but otherwise, Metacritic is nothing but a collection of reviews.

I'd look more at individual critics and why they say what they do, and Metacritic is a great launching point.

Fitocrop
November 17, 2011

I actually find Metacritic quite useful as a review warehouse of sorts. It provides you with the links to the complete versions of each of the Critic reviews it aggregates so you don't have to go around Googling "___________ review." It's all there for the reading. The Metascore (despite being its flagship feature) is actually the least important thing it provides.

By the way, Mega64 made a pretty funny video about this same topic :

http://mega64.com/2011/11/12/think-twice-about-reviewing-uncharted-3-modern-game-journalism-the-movie-premieres/

Dsc03881
November 17, 2011

I don't follow metacritic but I do read reviews from various sites.  Scores/grades have a little impact on me when it comes to purchasing games.  I think all comments shouldn't be allowed on a review until the game itself is release.  Yeah, we have forums but I think comments should be lock until release so that a fair response is given back.  Just vecause zelda got a 10 from different people shouldn't make a another reviewer jump on the band wagon.  We need to hear the opinions and thoughts from all with a fair mind.  Gamers need to play it and then blog/podcast their personal feeling and opinion about games.

Trit_warhol
November 17, 2011

@Roberto - That Mega64 video is nothing short of genius.

@Rus - I think the reviewers behind the Metacritic aggregate have a lot to answer for. I tweeted some reviewers regarding the perfect scores they either did, or said they would award Battlefield 3. They openly acknowledged that the games they were reviewing were flawed, but opted for a perfect score anyway. http://bitmob.com/articles/editorial-reviews-scores-dont-add-up

You can't assign blame to gamers alone. Dennis Scimeca wrote a similar piece which I think missed the point. For years, game reviewers seem happy to use one side of the 1-10 scale for the majority of the games they review. As above, a game assigned a 7.5 is seen to have bombed when mathematically, 7.5 is an above average score. 

Maybe reviewers need to acknowledge that a game that is average should get 5 or 5.5 instead of 7. A good game should get a 7 instead of an 8.5. Better yet, why not scrap scores altogether? 

Rm_headshot
November 17, 2011

At no point would I suggest Metacritic is evil (though you can absolutely take issue with some of their methods of assigning more subjective scores in the cases of alpha/non-scored reviews). It's a tool. It's a starting point. The problem is too many people stop right at that big number in the pretty colored box.

And it's also worth noting that publishers have gotten wise on how to game Metacritic. A lot of early, super-positive reviews go up -- possibly from sites you've never heard of -- to artificially inflate the aggregate during the critical launch window. Battlefield 3's a good example. I think it's score was around 94 the week it released (and I wouldn't be surprised if that number didn't show up in timed ads), but now it's settled down to a more reasonable 89.

Trit_warhol
November 17, 2011

If you look at those first reviews for Battlefield 3, you'll see that a lot of them were from big names: IGN, Gameinformer, Gamespot, Gamespy. There were some lesser-knowns in there, but plenty of established sites as well. 

I think you've identified that publishers and some unscrupulous writers/sites are also to blame. Publisher gives early access to sites that may or may not (I don't have any proof) agree to publish a more positive review. High scores are given and it is accepted by the peanut gallery that this is an exceptional game. Any site that breaks from that accepted norm (like Simon Parkin w/ Uncharted 3 and Tom McShea w/ Zelda: SS ) are seen to be giving lower scores as a means of grabbing attention. 

Gamers are expecting higher scores because - for the most part - that is all they are seeing. 

Default_picture
November 17, 2011

@Tristan-I believe you've (to use a video game metaphor) hit the ethnic stereotype right in the face.  This is why reviewers need to tone down their scores, and get rid of the "four point scale" of video game reviews.

Another point to make is the abundance of out of 10 reviews.  Simply put, 10 points are too damn much for a review.  If you would award out of four or five, like every other for of media criticism, none of this would happen.  Also, there would be tons of leeway for the reader and reviewer to voice their opinion.  

5211_100857553261324_100000112393199_12455_5449490_n
November 17, 2011

I've purposely never let reviews decide if I wanted to buy a game or not.  Or pre-release hype, for that matter; when Assassin's Creed came out, I'd seen videos and read stuff, and nothing I had seen interested me.  I heard "Prince of Persia's spiritual successor" (and mind you, I like Prince of Persia.  A LOT.) and saw the pretty graphics and said "Meh."  It wasn't until a good friend of mine told me how AMAZING the storyline was, and how they had cleverly interweaved fictional conspiracy theories believably into an actual historical timeline, that I decided I would give it a shot.  It ended up being amazing.

 

I've bought about eighty bajillion Sonic games and collections on all their platforms because I KNEW Sonic, as a series, always had at the very least PLAYABLE history with me (Sonic '06, you are retconned.  You never happened) so I knew when Sonic Generations hit the PC, I'd be picking it up, since that's where I got my first Sonic fix.  Sonic had built a relationship of trust with me, and I rewarded that with a chunk of my paycheck.

 

Reviews are fine.  They are opinions.  When they become not opinions, but something that affects an aggregate score (ESPECIALLY when those scores are manipulated; 3/5 does NOT equal 60%, nor does 4/5 equal 80% when taken out of context) and THAT is being contested, we have problems.  I like reading opinions, but if I had my way, we would just take numbers out of the equation and simply... I don't know.  READ reviews.  Imagine that.

Trit_warhol
November 17, 2011

Um... 3/5 does equal 60%. 4/5 does equal 80%.

For example: if I slice a pie into 5 pieces of equal size and eat four of them, I would have eaten 80% of that pie. I would also feel ill. I don't think using a 5 point scale allows for a different context either; it just means that you've cut the pie into less - although larger - pieces. 

Default_picture
November 17, 2011

I think Bryan's point was that each reviewer has his/her own interpretation of the numbers he/she uses to rate a game, and there are probably a lot of 'em out there that would only give a rating under 70% to a game that was just garbage, not worth playing; while others would say a 60% or 3/5 is a totally enjoyable game that just underachieves in a few areas.

I think it really depends on how much of an impression the American public school system has made on the reviewer's (or the reader's) mind.  The per cent scale (be it expressed as a 92/100 or a 9.2/10) brings to mind a grading system where anything under 75 is a failure, even though it's proportionally still pretty good.  And, to Bryan's point, measuring in fifths (or tenths) instead of hundredths makes it easier to avoid that "school grading" mentality, for both the reviewer and the reader.

Of course it's impossible to tell just from looking at the numbers whether or not the reviewer is affected by that mentality, so people should really place less importance on these ultimately arbitrary numbers, and more importance on the words that precede them.

Default_picture
November 17, 2011

Clearly, the internet is just too easy to use.

Default_picture
November 17, 2011

If only I was an EIC of a major publication or something.  I'd write a positive review for a good game, and then give it a 1.0 score, just to see what happens.

Default_picture
November 18, 2011

Great and timely article. I've only recently started to read the comments on reviews. I'm still amazed by the amount of people that declare a review wrong without having played the game. It's something I've hardly ever seen for, say, film reviews. Why? Because gaming breaks brains. Games are part of an Illuminati plan to dumb us down. That's the only possible explanation.

Default_picture
November 18, 2011

I feel this is a problem on several fronts though. On one side you have gamers who think a review makes or breaks a game. While it can help/harm sales/form opinions of people who will never play it. Reviews are just the feelings of said reviewer.

But this is a bit of a tricky thing though. Since a few people rely on reviews to tell them if a game is good, and whether to buy it or not. So a positive/negative review can help/harm new titles from new series people are unsure of.  But when it comes to established series where a fan base is going to buy the games regardless of reviews another person's opinion is  rather irrelevant. Unless the game is completely broken.

But really gamers should just go out there, and buy games, and make their own opinions. While it's not possible for most gamers out there to buy tons of games, and play them, and form a opinion on them. The least they can do is not argue talking points, and claim it as their own.

 

Now onto the other side of the problem. The reviews itself. Personally I feel when it comes to game reviews scores have been inflated to nearly comical proportions. We seem to be living in an era where a 9-10 is the only worthy score, and anything below that is considered 'trash'.  Perhaps gamers have gotten so use to high scores we now expect them on anything. Or perhaps its due to the idea of getting ones money worth.  If there was one thing I could change about gaming it would probably be fixing the review scale into the old EGM system.  Where a 10 should be impossible to get, and a 1 meant stay away, and a 5 meant the game is average and middle of the road.

But with all the review styles out there, from 1-4, 1-5, A+-F, 1-10, 1-100. The main trick is to get reviewers to utilize the scale more, rather than sticking the best game they played that month  with a 9-10.

Default_picture
November 18, 2011

The gamers who bitch and moan about a score that doesn't fit their own interpretation are probably only looking for validation, anyway. Think about it: If someone complains about a low (or high) score -- especially without having played it -- are they really using the review as a buyers guide, or merely to vindicate their own opinion?

Default_picture
November 18, 2011

You also have to consider that a good portion of the people responding to the critics are actually kids. (Yes I know a lot of adults also play video games and act like 12 year olds)

Kids nowadays are a lot more advanced with technology, its kind of common place with them.  I have 10 year old nephews that send me multimedia messages all the time with their cell phones.  Even though they are tech savvy, they still arent mature and in the end they are still kids and will act like children when the opportunity arises.

This doesnt excuse the adults behaving like children, but I have a feeling there are a lot more of them than you think posting on these gaming sites.

Default_picture
November 19, 2011

About the Gamespot review:

I haven't visited or trusted Gamespot in a while, which is why I come to Bitmob. But to each his own. If he thinks it's a 7.5, that's fine. 

But you do have to say and think -- Everyone else thinks this iteration of Zelda is awesome. Nearly every other iteration of Zelda has been reviewed as being awesome. So when you're the one guy that gives it a 7.5...you have to question it.

Maybe he's Roger Ebert. Maybe he's some great critic that understands the flaws better than mainstream society. Maybe he sees the imperfections of the Mona Lisa and thinks it doesn't deserve the praise that it is receiving. Or maybe he's just a guy with an idiotic opinion.

Either way, it's his opinion to have...so he has the right to have it and publish it. But I also have to right to surmise that he's probably just giving a solo idiotic opinion on the game.

Default_picture
November 19, 2011

This is exactly why we over at greatgamingcrusade.com have switched to a 5 point scale. Actually READING the review is what matters most. Also, having a 5 point scale makes more use of the full scale as those with a 10 point or 100 percent scale usually only use the top 1/2 to 1/3 of their scale anyway and this is why a 7.5 reads mor elike a .5 to most readers. It's a mixxed bag in my mind: the readers AND the reviewers are to blame equally.

Default_picture
November 20, 2011

This article mistakes the gamers it criticises for something that they really aren't: People who buy a game because of a review. - Off course there are many many people out there who buy because they see a favourable review but those usually ain't the ones who take part in discussions. The loud people usually are very well informed about their games because they tend to follow the development of games they have interest in, in very detailed ways. While they might not be able to get their hands on early builds and review copies, I say there is a great chance that those people actually know more about a specific game than a professional journalist because they passionately follow the development of a few games they are interested in over a multitude of different media outlets and don't have to deal with work while doing so, like a journalist does. In addition to that off course you can know if a game is good or bad if you haven't played it. I and many others have been gamers for a very long time and at least I usually know pretty well if a game is worth my while without having played it or seen a score. That's experience.

 

Here starts the specific problem the article deals with: While what a player sees might be enough for him to judge if he'd like a game or not, it is not objective criticism. And while those loud gamers don't really need objective criticism to judge if a game is worthy buying or not, they want it critics for entertainment and "becoming part of something". Owning a game that is appraised always felt like something good. Back in the day I knew from the first screenshots I saw that I needed to have Metroid Prime, I bought it day one and it had a sticker that said "96%" on it. That sticker was completely irrelevant to my purchase and looked really ugly but it is still on the cover because I liked the thought of owning something this good.

 

But here is the problem: The journalist doesn't write the review for me but for all people in his audience and mostly for the people who need his advice to know which game to buy. And if 80% of this audience can have fun with Uncharted 3 I cannot expect the reviewer to give it a shit score just because I wouldn't have fun with it. Et vice versa.

 

But please, stop giving 10's. That's just ridiculous.

You must log in to post a comment. Please register if you do not have an account yet.