Bitmob_vb_ Bitmob is now a part of VentureBeat
Separator

The Top Bombs of The Current Generation: Review Scores vs. Sales Data

Andrewh
Sunday, June 14, 2009

Editor's note: Andrew whips out the TI-86 graphing calculator again and turns its withering, number-crunching gaze on the biggest sales disappointments of this console generation. Armchair analysts will find some pretty fascinating stuff in here, especially in the genre breakdowns after the jump. -Demian

 

Concluding my series of articles on top bombs for the current generation of consoles, I will now round up all the titles to compare the biggest bombs from this generation. Using the AURA index ("Achieved Under Review Aggregate", click here for more info on this scoring system), I have balanced scores and sales data to look at the biggest commercial bombs that were critical hits.

A brief introduction, if you haven't seen my earlier articles. The better the review score, the higher the AURA score. The lower the sales as a ratio to a console's install base, the higher the AURA score. An example of how it works: When comparing two games with equal sales, the game with the higher review score will be the bigger bomb. Likewise with sales: When comparing two games with an equal review scores, the lower-selling as a factor of console install base will be the bigger bomb. And a final clarification, if a game has a higher review aggregate, it can sell more copies while still remaining a bomb, under the assumption that games with higher scores will sell more. The rule: the larger the AURA score, the larger the bomb.

Follow the links to see the previous lists featuring console-specific bombs: Top Ten Wii Bombs, Top Ten Xbox 360 Bombs, Top Ten PS3 Bombs.

The following list was created using data from gamerankings.com and vgchartz.com. The 228 games across all platforms that scored an 80 or above on gamerankings.com (an arbitrary line that denotes "consensus critical hit") were scored using the AURA system. And for the sake of continuity, I have two lists, one with sports titles, one without.

TOP 25 CURRENT CONSOLE GENERATION CRITICAL HITS/RETAIL BOMBS
25. MLB Power Pros - Wii - .69
24. Guitar Hero: Metallica - Wii - .71
23. Pinball Hall of Fame - The Williams Collection - Wii - .75
22. Virtua Fighter 5 Online - Xbox 360 - .82
21. Sonic's Ultimate Genesis Collection - Xbox 360 - .83
20. Okami - Wii - .87
19. Tales of Vesperia - Xbox 360 - .87
18. Eternal Sonata - PS3 - .98
17. Punch Out!! - Wii - 1.00
16. Viva Pinata: Trouble in Paradise - Xbox 360 - 1.18
15. NHL 2K7 - Xbox 360 - 1.24
14. Bully: Scholarship Edition - Wii - 1.25
13. The House of the Dead: Overkill - Wii - 1.26
12. The Chronicles of Riddick: Assault on Dark Athena - Xbox 360 - 1.26
11. The Chronicles of Riddick: Assault on Dark Athena - PS3 - 1.44
10. Zack & Wiki Quest for Barbaros' Treasure - Wii - 1.46
9. Mad World - Wii - 1.47
8. FlatOut: Ultimate Carnage - Xbox 360 - 1.60
7. Geometry Wars: Galaxies - Wii - 1.70
6. Pro Evolution Soccer 2008 - Wii - 1.81
5. MLB Power Pros 2008 - Wii - 1.88
4. Rune Factory Frontier - Wii - 3.91
3. Boom Blox Bash Party - Wii - 4.38
2. Klonoa - Wii - 5.35
1. Pro Evolution Soccer 2009 - Wii - 8.75

 

And here is the alternate list, excluding sports titles, a particular niche unto itself.

TOP 25 CURRENT CONSOLE GENERATION CRITICAL HITS/RETAIL BOMBS
25. Unreal Tournament 3 - Xbox 360 - .51
24. Pure - Xbox 360 - .56
23. Disgaea 3: Absence of Justice - PS3 - .57
22. Trauma Center: Second Opinion - Wii - .60
21. No More Heroes - Wii - .65
20. Guitar Hero: Metallica - Wii - .71
19. Pinball Hall of Fame - The Williams Collection - Wii - .75
18. Virtua Fighter 5 Online - Xbox 360 - .82
17. Sonic's Ultimate Genesis Collection - Xbox 360 - .83
16. Okami - Wii - .87
15. Tales of Vesperia - Xbox 360 - .87
14. Eternal Sonata - PS3 - .98
13. Punch Out!! - Wii - 1.00
12. Viva Pinata: Trouble in Paradise - Xbox 360 - 1.18
11. Bully: Scholarship Edition - Wii - 1.25
10. The House of the Dead: Overkill - Wii - 1.26
9. The Chronicles of Riddick: Assault on Dark Athena - Xbox 360 - 1.26
8. The Chronicles of Riddick: Assault on Dark Athena - PS3 - 1.44
7. Zack & Wiki Quest for Barbaros' Treasure - Wii - 1.46
6. Mad World - Wii - 1.47
5. FlatOut: Ultimate Carnage - Xbox 360 - 1.60
4. Geometry Wars: Galaxies - Wii - 1.70
3. Rune Factory Frontier - Wii - 3.91
2. Boom Blox Bash Party - Wii - 4.38
1. Klonoa - Wii - 5.35

The Great White Bomb

It is obvious from the list that a smaller portion of the Wii install base are buying good games on the Wii. In fact, of the 38 games that scored better than an 80 on gamerankings.com, 16 were on the first list, while 14 were on the alternative list, for a total of 18 combined, with nine of the top 10 appearing on the console. That's nearly half the "critical hit" games of the Wii, confirming what was suspected: good games are not succeeding on the console. [Editor's note: That last line might be a little misleading. It's probably more accurate to say that good games are not selling on the Wii to the same extent, relative to install base, as they are on Xbox 360 and PS3. -Demian]

You will notice that remakes, original titles, ports, casual games, hardcore games, and classic franchises are all included on the list, restricting the "bomb" title to no specific type of game.

Xbox 360 Bombs vs. PS3 Bombs

While both consoles had games on the list, only three on the alternative list were PS3 titles, an exclusive (Disgaea 3), a timed exclusive for the Xbox 360 (Eternal Sonata), and a multiplatform title, half of which is considered a remake (Chronicles of Riddick).

As for the Xbox 360, FlatOut was the only non-Wii game in the Top Ten. Five Xbox 360 versions of multiplatform titles were on the list, with only Chronicles of Riddick's PS3 version also making the list. Two titles to make the list were Xbox 360 exclusives.

What does this mean? I will go as far to say that a greater percentage of the PS3 install base are buying the good games. However, due to a greater install base, Xbox 360 good games outsell PS3 good games, an important fact, when the games cost the same on both systems. Despite greater attach rates for good games, there are better absolute sales figures on the Xbox 360.

Beyond the Bombs

While the focus of this and the preceding articles was to look at critical hits/retail bombs, plenty of data was amassed that was irrelevant to this end. However, that data does provide interesting insights into the performance of particular styles of games across the three platforms, based on multiplatform titles. I've divided the multiplatform games into 8 categories: FPS, Third Person Adventure/Action, Music, Sports, Racing, RPG, Fighting, and Other.

FPS

Of the FPS multiplatform titles studied Call of Duty 4, CoD: World at War, Far Cry 2, Condemned 2, The Darkness, Rainbow Six Vegas, and Rainbow Six Vegas 2 all performed nearly identical on all platforms (with the expception of the Wii version of CoD WaW scoring a .20 vs. the Xbox 360's and PS3's .03). Discrepancies were found with Battlefield: Bad Company, BioShock, Call of Duty 3, The Orange Box, Unreal Tournament 3, Chronicles of Riddick and GRAW 2. In all instances, the Xbox 360 version fared better, with the exception of Unreal Tournament 3 (.23 vs. .51). The largest discrepancies were Bioshock (Xbox 360's .13 vs. the PS3's .45) and The Orange Box (Xbox 360's .25 vs. PS3's .32)

In short, FPSs perform similarly on both consoles, with a slight advantage to the Xbox 360.

Third Person Adventure/Action

This category is a loose group of games, combined for the sake of convenience. Here, scores were similar, although when there was a discrepancy, the PS3 usually came out better. Such was the case for Dead Space (PS3's .23 vs. Xbox 360's .29), Devil May Cry 4 (PS3's .12 vs. Xbox 360's .17) and Prince of Persia (PS3's .17 vs. Xbox 360's .28). Discrepancies of .02 or smaller were seen with GTA 4, Lego Star Wars, and Resident Evil 5, where the PS3 came out slightly ahead in all cases. The only game to buck this trend was Saints Row 2, where the Xbox 360 version (.16) outdid the PS3 version (.20).

The largest discrepancy of all was with Bully: SE. The Xbox 360 version (.30) performed well, while the Wii version (1.25) can be considered a bomb.

Music

Guitar Hero 3 and Guitar Hero World Tour had nearly identical scores across all three platforms. The Metallica expansion fared less well, doing best on Xbox 360, then PS3, then Wii.

As for the Rock Band Franchise, there is an interesting turn of events. While Rock Band did the best on the Xbox 360 by a modest margin (.07 vs the Wii and PS3's .12), the PS3 version of Rock Band 2 did slightly better than the Xbox 360 version (.09 vs. .10)

Sports

In nearly all instances, the PS3 outperformed the Xbox 360 in terms of sports game sales as a ratio to the install base. This was especially prominent in college sports, soccer, hockey, and golf. Identical or near identical scores were seen with Fight Night Round 3, the Madden Series (Xbox 360 did better with 08 while PS3 did better with 09), the NBA 2k series. The Wii, where it had a competing entry, did poorly in comparison with the exception of Tiger Woods 09, which actually did better than both Xbox 360 and PS3. The only game that scored better on the AURA index on the Xbox 360 was Skate 2.

Racing

All multiplatform racing titles did better on the PS3 except for Dirt, which fared best on the Xbox 360. These titles include: Burnout Paradise (which is also available for download, figures not taken into account for the AURA scores), Grid, Midnight Club: Los Angeles, and Pure. Where the scores differed, they differed significantly.

I'm at a lost to explain this, other than Xbox 360 has its flagship racing titles (Forza and Project Gotham) on the market, while PS3 owners are waiting for Gran Turismo 5. I can also offer the explanation that since MotorStorm is out, it makes sense that Dirt would fare poorly on the system. To counter this argument, Pure did do better on the PS3, if only by a slim margin.

RPG

Only two mulitplatform titles that were included in this study were RPGs, both developed by Bethesda. Oblivion did better on the Xbox 360, but only slightly, surprising given how far in advance it came out (.08 vs. .12). This is much closer than other timed exclusives, BioShock in particular. Fallout 3 performed similarly on both consoles.

Fighting

Four fighting titles came out on both platforms. The PS3 versions had lower AURA scores in all instances, showing that a higher percentage of the install base is buying this type of game than on the Xbox 360. These were Soul Calibur 4 (.12 vs. .19), Street Fighter 4 (.12 vs. 23), Virtua Fighter 5 (Online for the Xbox 360) (.25 vs. .82), and UFC 2009 (.34 vs. 51). Up to this point we have seen better performance of PS3 titles over Xbox 360 titles. Despite this, Xbox 360 titles sell more, if to a smaller percentage of the audience. Virtua Fighter 5 for the PS3, without the online component actually sold 90k copies more than teh Xbox 360 version. This is a rariety found in the data.

Other

Finally, only two games that couldn't be classified elsewhere were released and scored above 80 on gamerankings.com. Civilization Revolution fared slightly better on the PS3 (.28 vs. .34) while Sonic's Ultimate Genesis Collection did much better on the PS3 (.40 vs. .83). This could point to the fact that PS3 owners tend to be more "retro" centric. Alternatively, it could point towards differences in online offerings.

Conclusions

The data has offered a variety of trends from which we can draw conclusions. First, the Wii has fewer games that have garnered critical acclaim. Despite this, these games are faring the worse in terms of the install base.

PS3 games tend to do better than Xbox 360 titles when sales are viewed as a percentage of install base. The only category in which this trend is reversed is with FPS games. Sports games do slightly better on PS3, while racing and fighting games do much better on the PS3.

With these trends we can identify the character and nature of the customer base for each console. We can also point to the fact that the PS3 has a narrower selection of games that have been praised in the enthusiast press. This may have concentrated sales to a smaller selection of titles, thus driving AURA scores down, while Xbox 360 owners have more games to select from when making a purchase decision.

While the scores were lower for the PS3, this only means that they "do pretty good for what they have." Simply as a function of install base, the Xbox 360 version of games will out-sell PS3 games, with rare exceptions. And where games are priced similarly, Xbox 360 titles stand to bring in more revenues, and thus more profit.

As a basis for development and publishing decisions, the AURA system would fare poorly. But as a part of a conversation about those classics that we love and hated to see bomb at retail, the AURA system has provided insights.

Especially for us poor Wii owners.

 
Tags: sony, Nintendo, Wii, Microsoft, PlayStation 3, Metacritic, bomb, Review Scores
Problem? Report this post
ANDREW HISCOCK'S SPONSOR
Related Stories
 
Comments (18)
Add a new comment
Default_picture
Clint Carr
June 15, 2009
You're not factoring in budget or publisher expectations, so you're putting the Wii games in an unfair position.
Default_picture
John Michael
June 15, 2009
Andrew, you're a beast! Great work, as usual.
Andrewh
Andrew Hiscock
June 15, 2009
You could say that the developer's are putting the Wii games in an unfair position. Rimshot! But seriously, the budget and all that can't be taken into account, but that info is not readily available. I did mention in the previous installments, specifically in the case of Sonic's Ultimate Genesis Collection that despite a poor showing it probably pulled in a profit. But as for Wii games, the numbers reveal that these games are at a disadvantage (but that argument has been there since the N64, truth be told) based on historical purchase trends, etc. etc.) This just emphasizes what we already know about the Wii, that things [i]are[/i] different, and that every worry prior to the release of Zack and Wiki, MadWorld, and Punch Out!! are totally sensible worries. What the numbers do say is that developers of Wii titles should trim budgets and lower expectations.
Default_picture
Clint Carr
June 15, 2009
Zack and Wiki was under promoted, under shipped, and had low expectations. Punchout only has 2-3 weeks (of the slowest month in the year) of sale numbers to go on (and to be honest, Wii games have more legs than Xbox360 games, so there's a good chance Punch Out will go on to sell great). Madworld is a niche game that was thrown on a unreachable pedestal. Out of those, Zack and Wiki is the only one that's been out for more than a year, and it sold over 250,000 worldwide. Capcom has gone on record saying it exceeded expectations, so is that REALLY a bomb? What does it take, sales wise, to not bomb? Halo and Wiifit numbers?
Demian_-_bitmobbio
Demian Linn
June 15, 2009
Keep in mind that this definition of 'bomb' is also closely tied to review scores. So a very highly-reviewed game has to sell a lot more in order to not be considered a bomb under these criteria. If you want to judge bombs strictly by unit sales, many of these games wouldn't qualify.
Default_picture
Tiff Chow
June 15, 2009
I just wanted to say that seeing that screen shot of the Zack and Wiki Frog Bomb just about made my day. Thank you.
Andrewh
Andrew Hiscock
June 15, 2009
Hey, this is what happens when you use an index. Numbers mean different things to different games, depending on the circumstances of its creation. Again, I'll refer you to my first installment where I explain the scoring system: "The scores can represent gaps in marketing, region, market demographics, or any other factor that lead to a game's failure. Also, it may not represent failure, as there is no accounting for developmental costs (impossible to discern). A high bomb score may have been a profitable game." But there is a standard, and sales will normally follow it. If Game A scores X and there Y Amount of Consoles out there, then Game A should sell Z copies. As for "sales wise not to bomb", depends on the game, and in this system, the score, BUT in specific for a Wii game in the mid 80s, you're talking 250-300k range, clearing about 1% of the install base? Halo sold to 41% of the install base, while Wii Fit sold to about 35%.
Default_picture
Raymond Fernandes
June 15, 2009
I'm not sure I like your AURA scale. In your own words,"When comparing two games with equal sales, the game with the higher review score will be the bigger bomb.", but what happens if that number is a decent number? What if it met developer expectations? What if 2 games sold a million copies, one got a Metacritic score of 60 and the other 85? By your scale, the latter would be labeled a bomb even though it sold perfectly fine. In a perfect world, all critically acclaimed games should sell millions, but this isn't the case.
Andrewh
Andrew Hiscock
June 15, 2009
Not necessarily. Even at lower levels you would see a difference, the 60 scoring even lower... out performing, as it were.
Img_1019
Aaron Thomas
June 15, 2009
A very thorough job analyzing a fascinating subject. It seems that people would rather nitpick than actually discuss the results. Andrew's not claiming that his system is perfect and he's not claiming there are no other factors at play. Ah well. I appreciate the fresh take and the obvious effort that went into this.
Default_picture
Ben Alford
June 15, 2009
I can't believe this article was posted on the front page. Your source of sales data is vgchartz, which is notoriously inaccurate. Your use the term "bomb" makes it read like an uninformed neogaf sales thread. The success or failure of a product isn't something you can draw up a formula for. It is something that must be looked at case by case, factoring in more data than you use--data that is extremely hard to come by, something you acknowledge. Also, you don't account for how long a game has been released. In your, "TOP 25 CURRENT CONSOLE GENERATION CRITICAL HITS/RETAIL BOMBS" list, look at the titles that are on the top and look at their release dates. Of course a game that hasn't been released for very long isn't going to have as high as sales. A majority of those titles have not been on retail shelves for more than a year. You cannot try to draw trends from statistically incomplete and inaccurate data.
Andrewh
Andrew Hiscock
June 15, 2009
You make some good points Ben. First, the use of bomb is to get readers. Vgchartz is inaccurate, but NPD is not wholly accurate either. As good as the data source is, I suppose, and I relied I the most readily available. Vgchartz does use NPD though. I acknowledge that this doesn't account for success or failure. More a "relativistic success quotient". It's a comparable index, not a profit margin. Time is an interesting issue. It's something I would really like to look at, factor a model into the equation. Wii sales graphs are very different from what we normally see, so developing a model that accounts for both types of sales curves would be difficult. But a sharp eye can filter data, make some comparisons and say "Okay Punch Out!! didn't do so hot, but that's early, but Geometry Wars Galaxies really stunk it up, eh?"
Default_picture
Clint Carr
June 15, 2009
Geometry Wars Galaxies was a port of a game that millions of people already played, and honestly, I doubt it cost much to develop or promote. Ports are hard games to sell, since people have already played them, and their hype is already gone by the time they're released. It's a bit unfair to compare Geometry Wars (Wii), Eternal Sonata (PS3), Okami (Wii) and to new and original games.
2
Joey White
June 15, 2009
Okay, the system isn't a perfect representation of sales success, but it's interesting, nonetheless. You mention that (by your system) the PS3 has fewer bombs than the Xbox 360 and Wii. The number of "bomb" games seems to be inversely proportional to the install base of the system. I wonder if you've considered that this may have something to do with the percentage of casual gamers on each system. The Wii install base is chock-full of casual gamers, after all, while the PS3 seems to largely be owned by the hardest of the hardcore. We hardcore gaming nerds, after all, are theoretically more discerning in what we pay our $60 for. We pay more attention to reviews, and we like games with a little bit of depth to them. Casual gamers, on the other hand, are less likely to read reviews and more likely to base purchase decisions on colorful box art and persistent GameStop employees. And rather than playing games with depth, they're more likely to prefer something they can wave their arms around to for 10 minutes before turning the system off and getting on with doing the laundry or whatever it is that nongamers do while they're nongaming.
Default_picture
Brian Shirk
June 15, 2009
That reminds me--I need to get Klonoa when I have some more cash. A year ago, I would have had plenty of money for all these Wii exclusives, but now it's getting hard to afford them. Unfortunately, I think the timing of some of these titles influenced their poor sales.
Andrewh
Andrew Hiscock
June 15, 2009
@Joey. You bring up an interesting point, something I wanted to address, but was unsure of. Why the number of bombs is inversely proportional to console sales.. Well, I think you nailed it on the head. Casuals come in after the fact, and I wonder if the Xbox had similar numbers when it had only sold 8 million consoles, the more casual (or less hardcore, I should say) picking up the other 9 million consoles. That said, I wonder why PS3 seems to do better with sports games, a type of game I would say is the bread and butter of the less hardcore fan. It's weird, there are two pieces of evidence working either side of the argument.
Demian_-_bitmobbio
Demian Linn
June 15, 2009
Many of the hardest of the hardcore that I know opt to buy multiplatform games on PS3...hdmi, etc. Maybe that factor is coming through for sports games.
Default_picture
Allistair Everett
June 16, 2009
Great job Andrew, but I think there is an oversite in your research: the cost it takes to make a game. By your algorithm, Shenmue would be a success but when you factor in that it took like about $70 million to make, then it was a giant bomb (.com?) Likewise, I think titles like Deadly Creatures that spent a lot of money acquiring a star studded cast would have a bigger spot on your list. On the other end, Gears of War which only cost a stunning $10 million to make is one of the generation's biggest success stories because of it.

You must log in to post a comment. Please register if you do not have an account yet.