Metacritic now rates developers, sliced bread to be aggregated next

Twit
Tuesday, March 29, 2011
EDITOR'S NOTEfrom James DeRosa

I am from the future. I am a Robotic Categorization and Aggregation Unit. My designation is 4A539. I have come to rate your overall quality as a human being. Please submit yourself for review.

[Update: Metacritic has yanked the developer ratings, though they still list the scores for each of a developer's games. The information is still there, but it's no longer automatically averaged for Metacritic users. Bust out a calculator, and rate your favorite developers for yourself!]

Metacritic, the website that is the bane of many professionals in the video-game industry, now has one more thing to make people nervous: It's begun rating the back catalogues of individual developers.

Type a well-known developer's name into Metacritic's handy search bar, and you'll see a profile with all of the person's work, how their games have scored, and a list of all their average scores from which you can theoretically infer an individual's overall success. Of course, we're working with averages here, so not all the scores seem to tally up how you think they would.

For example, I tried searching Denis Dyack. Dyack's reaction to the mixed reviews of his last game, Too Human, was a bit of a debacle. His profile only lists three titles and their scores: Eternal Darkness: Sanity's Requiem, Metal Gear Solid: The Twin Snakes, and Too Human. This is odd because he created the Legacy of Kain series, which doesn't have a score. As it stands, he's got a couple nice, green numbers, with only Too Human reflecting poorly on his body of work. But if you search someone as prolific as Shigeru Miyamoto, you'll see a profile that is much more colorful. Some of the entries list him as a "General Producer," which makes one wonder how involved he could have been in two titles released less than a month apart.

 

Many industry insiders express some hesitation about Metacritic and believe in more elaborate and comprehensive ways of talking about video games. They explore questions like who the developer geared the experience toward or how tip-top presentation can work to cover up a few glaring mechanical problems. Metacritic is only interested in cold, hard numbers, and that sometimes creates problems for reviewers who use different scales. 1UP, for example, uses a grade-school letter system for the sake of clarity. Meanwhile, in EGM, a "5" is average, but in Game Informer, a "7" is average.

Personally, I think it sounds like Metacritic is on its way on categorizing everything like some sort of hyper-efficient, humorless, bureaucratic automaton. Hopefully, in a few years time, I'll be able to find out whether or not my morning toast is an irresistibly delectable "87" or a bland, boring "62."

What do you think of Metacritic's new rate-a-developer feature? Do you think aggregation works, or is it too reductive?


This story was cross posted on HB Hud [via Gamasutra].

 
Problem? Report this post
MARCEL HOANG'S SPONSOR
Comments (8)
Photo-3
March 29, 2011

This is why UC Santa Cruz favored written reviews of the students performance in classes over arbitrary letter grades. The problem came (and the reason why they eventually switched over to traditional letter grades (though not completely)) when people wanted to apply to grad school and needed a GPA.

How do you suggest these video game designers get into grad school without this aggregated scoring system?!

Robsavillo
March 29, 2011

Metacritic actually [url=http://www.gamesindustry.biz/articles/2011-03-29-metacritic-disables-developer-career-scores]pulled[/url] this feature today due to criticism (based on inaccuracies of the scores).

March 29, 2011

The one part I did like about this system, was that I could type in the developer and see all of the games they had a hand in.  Really showed how long some people have been around for. Giving a person an overall ranking isn't so nice and I expected it to be removed quite quickly as Rob pointed out.

Default_picture
March 29, 2011

If they are going to implement a feature like this they should review development studios, not individual developers. The work of one individual on a specific game does not mean the developer is bad or good, that is an unfair way to judge a single person.

Pict0079-web
March 29, 2011

Holy cow, that's not even fair for developers. Yeah, I can understand that Metacritic would eventually pull this. Maybe if they tried a less personal system of rating, such as imdb, it might end up more fair. However, this is Metacritic, the company who doesn't even keep a database of games older than 2002.

And yeah, this is an awful way to judge an individual.

Twit
March 29, 2011

Thanks Rob. That explains why I didn't see an overall score despite reading the story for my first time on Gamesradar and checking it out myself. They kept referencing the score but I didn't see any average.

Robsavillo
March 30, 2011

Even though Metacritic pulled the aggregated developer scores, this feature will likely return once the site fills in the holes, as you mentioned in the article, currently in their databases.

Img_20100902_162803
March 30, 2011
Does anyone know how metacritic picks the publications that they track?

You must log in to post a comment. Please register if you do not have an account yet.