Should reviewers wait for patches?

Default_picture
Friday, May 13, 2011
EDITOR'S NOTEfrom James DeRosa

Tim thinks it might be a good idea for reviewers to wait for developers to patch major problems before casting final judgment on a game. He's isn't certain this idea is a good one -- and I'm not either -- but it could be worth a shot!

Brink is out in the U.S. -- and has been for a couple of days now -- but what about over here in New Zealand? Nope. That means I’ve had plenty of time to digest the reviews and the subsequent reactions. And because I’m not playing Brink right now, I am constructively twiddling my thumbs.

What do we want from the gaming press? Over time we’ve built up preconceptions about what a review should look like, and as long as it ticks the boxes, then we’re fine. If it doesn’t? It's time for Operation Tantrum. So what happens when a reviewer flat out says he won’t review it?

Face it, we’re a very demanding audience. We expect writers to review from the perspective of the average gamer, a mysterious creature from whom all our tastes have descended. We also often assume that this mysterious creature is us.

More importantly, we want these reviewers to get their reviews in on time because we pay their damn wages -- somehow -- through us buying games or economic trickle down...and stuff.

 

Anyway, I’d love to see more transperancy in reviewing. Reviewers shouldn't let the community’s expectations hold them hostage. Some reviewers have moved away from numbered scores, and this is good...sometimes. But a number can also help, and often does. The problem arises when the audience fixates on the number or a single line of the review.

Reading an article that says “look, this game’s not in a fit state to play right now, so I’m going to give it the benefit of the doubt and come back to it when it’s not playing so poorly” was far more helpful than reading others that said it was a buggy, unpolished mess not worth my time.

This way we don’t write the game off; we just acknowledge that a launch-day purchase might be underwhelming, so put it on the back burner for a while. It also acknowledges the power patches have to right wrongs that a launch review would have you believe are permanent.

Admittedly, there is the question of where you draw the line between minor launch fixes and ongoing patching. I’d hate to increase the workload for reviewers, but maybe sometimes we need a second wave of reviews one month in, or at least a quick article revisiting the bugs and assessing whether or not the developer has fixed them.

Cast your mind back to another somewhat buggy release: Alpha Protocol. I picked it up this year, and I enjoyed it. But it’s still kind of a mess and really only warrants a discount purchase. I was looking forward to it prior to release, but when review time came, it was found wanting. I didn’t buy it because the critical response influenced my judgment. I’m fickle like that.

To speculate, what if more people had said that it had promise, but advised waiting for a patch? Would Obsidian and Sega have less concerned its low Metascore and actually put resources toward a much-needed patch? Maybe Alpha Protocol would be better experience today than the post-launch orphan it became.

Unfortunately, the flip side is that this might give developers and publishers even more leeway to boot an unfinished game out the door with assurances that there will be a day one patch. I hope this isn’t the case, but I’m not an industry insider, so I don’t know. Cynicism tells me it’s pretty plausible, though.

It does seem to place the onus on the developers. Rather than giving a low score and crying “for shame, sir!”, could we instead raise an eyebrow and say “you know better than that, and I expect it back here, fixed, this time next week”?

 
Problem? Report this post
BITMOB'S SPONSOR
Adsense-placeholder
Comments (19)
Robsavillo
May 13, 2011

I think returning to games months later by the press is a great idea for several reasons. As you point out, we can see if any issues were fixed in a subsequent patch. But we also keep the title in the conversation longer...could that reduce the heavy front-loading of sales in the first few weeks? Maybe more people would decide to go back and look at a game they passed up. And we might get out of this chaotic preview cycle of continually fetishizing the "next big game."

Img_0020
May 13, 2011
I'm not sure about that. The case with Brink is that the released game is unfinished and that's really it. If Splash Damage were really concerned about the quality of their game they should have delayed it. We can't just reserve our judgement for the game until the future because we aren't sure about the future. Will the game be patched or not? And if so when will they patch it? It all just unknown. And telling consumers that this game is not good is of great importance. It's probably a good idea if the reviewer thinks that the game can be improved in a future update to write that in the review, but that shouldn't change their opinion of the game. Though if the game does indeed improve significantly, then I would like for that to be noted somewhere, maybe an amended review. But I doubt many will pay attention to that and really thats just the price their paying. the important thing is that we shouldn't withold a review for a later time just out of the possibility of some kind of improvement.
Avatar
May 13, 2011

I also believe returning to a game somewhere down the line after a patch is released is a good idea, but not at the expense of that early initial review. Timely reviews are just part of the business and people want to know what the deal is with the hottest new game now.

If it's buggy, I think a detailed review telling WHAT'S buggy about it is more helpful than just saying "This game is buggy, come back later please." Some of the bugs might not bother some people, so a detailed review from the get-go would still help them out. If a patch comes out later and fixes things then great, say what it fixed and how it makes the game better (or worse?).

Itsame_
May 13, 2011

Having played Brink (sorry Tim! >_<), I feel like you may be onto something. Them game feels odd. Besides the bugs and various network issues, the game modes... I feel they are lacking. An update adding some variation though; that would redeem it for me.

Really it's the Best/Worst thing about this generation; games and developers get second chances.

Default_picture
May 13, 2011

Patching games after release to fix any detrimental bugs is a practice that has become all too common these days, and like Badr said, if the game isn't finished, a delay may be the best option.

But, as Rob said, it could be a perfect reason to return the spotlight to games that weren't quite "there" yet at release date. 

If anything, it would give those good-but-not-necessarily-great games a better shot at actually making a splash, rather than disappearing into a realm of obscurity and whispers. 

So, as Tim mentioned in his article, I'm all for the idea of a re-review of a buggy game, after patches have been applied and bugs have been fixed. It's a great way for reviewers to be dynamic in their opinions and thoughts on video games, rather than static with a fixed numeric score.

And, on top of all that, it's fair to the video games, which is always awesome.

Default_picture
May 13, 2011

Oh God, excuse me for the mistake here, but I accidentally accredited this article to editor James DeRosa. I meant to place Tim Henwood's name in mention of the article's author. My bad...

Jamespic4
May 13, 2011

Fixed!

Vanilleavitaryt
May 13, 2011

I don't think it's fair to consumers to wait for a patch before reviewing the game.  If the reviewer is going to wait, then I think the developers should have waited and gotten it on the disc.  For a game like Brink, it may not be a huge deal to have a patch since the whole game revolves around online play already.  But what about single player games?  What if someone who isn't connected to the internet purchases a game and it's broken?  

The score should always reflect what's on the disc.  If people want to do a follow up review where they add a new section and an amended score that doesn't get published on Metacritic, I think that's a great idea.

Ironmaus
May 13, 2011

This is a good point. While a book won't change after publication and can be totally digested by one person in a limited amount of time, games change, and reviewers are commenting on the experience of playing even though they try to frame their comments as beign about the games themselves. Revisiting a game after one or two months—call it the "check up"—would be valuable both for seeing the improvements that have been made since release and the chinks in the armor that have been discovered. I'm sure patches will bring improvements, but having a bigger pool of diverse players for multiplayer will also improve the experience. On the flipside, something important may turn up when several million people play a game that might not have been noticed when a few hundred reviewers did their pass. The check up allows reviewers to give a more complete assessment of the game as it exists in the wild. I don't mean to get over excited, but this could be a big step toward more serious and respected games journalism.

100media_imag0065
May 13, 2011

I suggested something very much like this once on this site in a mailbag section. I said reviewers might want to think about revisiting games when a significant patch enhances a game, or makes it worse. Knowing how work can pile up, I know that this is unlikely.

Dan Hsu sure as hell hated my suggestion about this though, since he went on a mini angry rant for some reason. Hit a nerve maybe?

Default_picture
May 15, 2011

Cheers for the feedback, having now spent the weekend dipping in and out of 'mingleplayer' Brink, I'm a bit more informed as to where the original reviews were coming from. While the netcode needs some work, and work is underway, I've yet to hit any bugs that break the immersion severely. Then again, for this game, netcode is sort of the spine, so it really would be nice if they get that cleared up.

Alex and Matthew, you've hit the two points that I think make a follow-up helpful :

A) The original review should be able to objectively state what's wrong - not "it's too buggy for me", but "there are bugs, and let me tell you what they are". That way, you have a starting list of things to fix, from where you can build your comparison review later

B) We're dealing with a medium that can evolve, and do so rapidly.

A launch day game may play very differently to the same game a month on - so why not make room for this? Maybe not every game, but if a game has significantly changed, maybe go back and revisit it? Not exclusively for the ones that were messy on launch day, but for any game that has been substantially altered through a post-release patch - rather than just leave it up to forum-folk to argue about new weapon-balancing etc. 

From a sales perspective, I'm wondering how lost the cause is - I'm backing this because I'd like publishers to be encouraged to keep working on the game and - where multi is involved -  to keep the community active. But I understand that the launch week metascore is a big decider on the success, can game sales benefit  from a resuscitation a month later?

Trit_warhol
May 15, 2011

Great read, Tim. I wonder where we draw the line though. Especially if you look at titles that continually delivered content at no extra cost (for example: Team Fortress 2 and Burnout Paradise). Would we update our views for every title update, or do we use a time limit and talk about any improvements implemented during that period?

Gamespot employ a feature called "After the Fact: Reviews Revisited," where they outline additional content and features; however, to my knowledge, no actual review scores have been changed. In the current climate it may be well worth considering further in-depth consideration of games after their initial release. 

Picture_002
May 16, 2011

I understand in places where a title may release later than others that concern, but I'm not sure there's ever a really good excuse to delay a review for the people that are getting the initial review. End of the day, the review should reflect what the product is upon release, and considering there are still plenty of people not with constant online connection if any of all (a reality oft overlooked by regulars in online forums for understandable reasons) not everyone's getting that patch initially. So I don't think changing the intial review is a good idea.

I'm always for revisiting a game for more discussion as Rob says. If there's anything I've not cared for it's how quickly games fall out of regular discourse after release because of the preview heavy nature games coverage has always seemed. Heck for as horrendously overrated as I felt Heavy Rain and Alan Wake, I'm not so great it was a good thing that nearly a month or so after the reviews hit, some people seemed to forget they ever came out unti; year end award discussion hit in December last year. Whicb surprised me considering how much critics talked up their siginificance leading to and at release. Sure it's a good excuse to revisit patches and what issues may have been fixed, but frankly I think it's healthy every once in a while just to let a game simmer for a few months and revisit our initial reactions and reassess our in the moment thoughts of them for a multitude of reasons.

Default_picture
May 16, 2011

Fair point on not changing the initial review.

Agree completely with you on revisiting to encourage discussion to continue - this is kind of what I'm after. Enthusiastic forums don't seem enough these days to warrant post-launch support (looking at you, Alpha Protocol).

Also agree that it's worth doing for more than just patches.

Default_picture
May 16, 2011

NO revieweres SHOULD NOT WAIT!  Developers should take the time to thoroughly test their games!  It is great that we now have systems that can patch and update games (if you are connected to the internet) BUT this should NOT be the norm.  I didn't need any patches for Super Mario 1, 2 or 3.  If developers don't want reviewrs to tear up their games they should spend some extra time making sure they won't.

Default_picture
May 16, 2011

Reviewers shouldn't wait for patches before reviewing a game for the simple reason that many people use the review to point them in the direction of which title to buy. So, without the review to guide them, many gamers would buy a game based on the flashy commercial and then balk at the buggy mess they receive. Reviews are supposed to help you gauge what titles are worthy of your purchase right now, not which titles will be awesome a couple months from now.

Default_picture
May 16, 2011

Ok, so flat-out waiting is out, by the looks of things. But if you read the Ars Technica review, what really was the difference between that and a 'scoreless' review, if you took away the part where he said he'd wait?

I'm not saying wait because maybe the game will end up being awesome - Most of the time they won't be, even with a patch. I'm looking for a way to save games from that first week blow that a string of low scores can bring. See if there's a way to mitigate the damage, and save the game. Not to save the developer's hide, but to ensure that the fans of that game still have something to play in six months time that hasn't been abandoned to its bugginess.

I get that it's principled to say don't wait, and that's fine. I appreciate release day reviews too. But if we're taking that track, what's wrong with revisiting it off the back of that original review? I don't really see the harm there.

Also, I think it's unfair to expect developers to hold a game back -  if the publisher can see a game that looks finished enough to release, the developer is going to have a hard time convincing them that they'd like to delay the release date for 'tweaking'. Delays hurt the bottom line too, I'd expect. 

Default_picture
May 16, 2011

I agree that reviewers should revisit their reviews from time to time, but we shouldn't give developers the crutch of releasing buggy games' knowing that the reviews will take patching into consideration. I just think reviewers should review as is. On the whole though I see and agree with your point.

Lolface
May 16, 2011

I don't think reviewers should wait for patches to review games. Instead, I think that we, as gamers and customers, should demand better, thoroughly tested games from publishers and developers. If a game needs a patch within the first week of release, it was not ready to be shipped.

You must log in to post a comment. Please register if you do not have an account yet.