Metacritic Scores vs. Sales: 2009 Holiday Season Analysis

Robsavillo
Wednesday, April 28, 2010

MetacriticRecently in GamePro, Julian Murdoch examined the reviews aggregator site Metacritic. His story opens with a rather startling observation from Activision Vice President of Marketing Robin Kaminsky at the 2008 Design, Innovate, Communicate, and Entertain (DICE) Summit. The opening slide to her presentation read: "For every additional five points over an 80 percent average review score, sales may as much double."

Incredible! If true, game sales must increase exponentially as they score above an aggregate of 80 on the website. Surely, an employee of Activision -- let alone a VP of marketing -- must have solid data to back up such a bold declaration.

But I'm the skeptical type, so I did a little of my own research. I’m here to tell you that Kaminsky's statement is a load of horse shit, and that publishers' insistence on a correlation between scores and sales is unfounded.

 

Murdoch’s article, which is well worth reading, focuses not on Metacritic’s influence (or lack thereof) on sales, but on the connection that publishers make between scores and sales. In other words, he investigates the shady practice of tying developer compensation to review scores.

My goal, however, is to see if any meaningful correlation between Metacritic scores and unit sales exists at all. When I previously studied this issue, I concluded that the former have virtually no influence on the latter. Critically acclaimed games, like the Wii’s Boom Blox: Bash Party (which received a Metacritic score of 86), sold poorly, while much lesser-scoring games, such as Night at the Museum or Terminator: Salvation, sold significantly more.

My sample size for that examination was small -- I used May 2009 releases and 10 weeks of their sales data. To investigate this matter further, I decided to collect a much larger, more representative sample. My analysis for this article includes everything that came out between October 1 and December 31, 2009.

For each title, I collected 10 weeks of sales data as it was posted on VGChartz. Any releases that did not have the full 10 weeks and a Metacritic score were not included. Out of 364 titles (including all system ports between the Nintendo DS, Nintendo Wii, PlayStation 2, PlayStation 3, PlayStation Portable, Xbox 360), 218 made the cut. PC, iPhone, and digital releases were not included because of the lack of sales data.

Read on to page two for the breakdown.

 
1 2 3 4 Nextarrow
Problem? Report this post
ROB SAVILLO'S SPONSOR
Comments (9)
Default_picture
April 28, 2010

Really Sing Star Abba for the PS3? Why would you show that off in your collection?

Twitpic
April 28, 2010

@ Rich: Having you ever wanted to be the dancing queen? I know sometimes I get the urge.

Rob, really great and informative article. Taking Kaminsky's quote at face value seems to make sense, but I'm glad you did the dirty work and proved otherwise. Still, I wonder if Metacritic scores directly affect the developers pay. Maybe you could do a follow-up on that, hmm?

Brett_new_profile
April 28, 2010

SingStar Abba, two copies of a Splinter Cell game.... You know, I think that may be Shoe's game shelf.

And Rob, really really fantastic stuff. I hope to see more of these kinds of articles from you!

Mikeminotti-biopic
April 28, 2010

Awesome story, Rob!

Default_picture
April 28, 2010

Wow! Great story! It's nice to have a new perspective on review/sales relationships when you have the facts laid out in front of you like this.

(Not to be rude, but you should know the link to vgchartz.com on the first page is broken; it's missing a letter.)

Default_picture
April 28, 2010

This was a great article!!

Demian_-_bitmobbio
April 28, 2010

This very interesting. I'd love to see it as unit sales per metacritic score, normalized for the number of games with that particular score.... I think that would give you a very different result, considering the huge sales and relatively high scores of your three outliers.

Jayhenningsen
April 28, 2010

Great work, Rob. I love some good number crunching from time to time.

Default_picture
May 13, 2010

hi Rob! I would like to show you a info i've made based on your article. its here: http://www.badongo.com/pic/9381925?size=original If you like to talk to me, my email is ariel.tonglet (at gmail.com) o/

 

 

 

 

You must log in to post a comment. Please register if you do not have an account yet.