Separator

The Trials and Tribulations of a Video-Game Tester

Andrewh
Saturday, February 27, 2010

Editor's note: Every industry has its 0-level grunt. In media, it's frequently the editorial assistant. In restaurants, it's the dishwasher. In gaming, it's the tester. Andrew interviews a friend of his who worked as a game tester. Think it's a glamorous job? Read on to find out. -Jason


Full disclosure: The author has been a good friend of Steve Sutton for a very long time and was thus able to learn his story and secure the interview.

Play games for pay? Work in the gaming industry? Testing games sounds like a dream job, right?

It's only a dream job is if you enjoy monotony and have a severe lack of creativity, no independent spirit, and zero ambition to manage yourself. So claims erstwhile game tester Steve Sutton.

We all hear great stories about how so-and-so started at the bottom and worked his way up to become a world-famous video-game producer or where the insightful journalist makes the game of his dreams. But what about the stories you don't hear? The soul-crushing grind of video-game development? The shoveling of budget titles into bargain bins?

Sutton, currently a graphic arts and 3D animation teacher, spent 3 years in the development houses of Toronto. Incredibly skilled and creative, he left the Big City and returned to Newfoundland as a jaded and cynical former industry insider.

In his own words, this is his video-game story.

Warning: Sutton has very strong, sometimes offensive, opinions. They are wholly his own.

 

Andrew Hiscock: Briefly describe your work as a tester: the companies you worked for, the games you tested, and the duration of your testing career.

Steve Sutton: I worked as a game tester in Toronto between 2004 and 2007. The first company I worked for was Groove Games, an upstart game publisher that had a short history of low-budget first-person shooters. I was hired with their first test team, which they hired for development of their first-ever triple-A title, Pariah (developed by Digital Extremes).

In 2006 I went to DreamCatcher to work as a tester. DreamCatcher owns The Adventure Company and budget brand Silverlight, I tested games released under all three labels during my time there. This is the game list that appears when you type my name into MobyGames:

    • Dead Reefs (2007), The Adventure Company
    • Genesis Rising (2007), DreamCatcher Interactive Inc.
    • The Sacred Rings (2007), The Adventure Company
    • Agatha Christie: Murder on the Orient Express (2006), The Adventure Company
    • Evidence: The Last Ritual (2006), Ubisoft Entertainment SA
    • Hans Christian Andersen: The Ugly Prince Duckling (2006), PAN Vision
    • Secret Files: Tunguska (2006), Deep Silver
    • The Secrets of Atlantis: The Sacred Legacy (2006), Nobilis France
    • Secrets of the Ark: A Broken Sword Game (2006), THQ Inc.
    • WarPath (2006), Akella
    • World War II Combat: Iwo Jima (2006), Groove Games
    • World War II Combat: Road to Berlin (2006), Groove Games
    • Combat: Task Force 121 (2005), Groove Games
    • Land of the Dead: Road to Fiddler's Green (2005), Groove Games
    • Pariah (2005), Groove Games

A few more either weren't added to MobyGames or were canceled before they were released.

AH: What did you do during the average work day? Did your average work day change from project to project?

SS: The average work day for a tester depends on what stage of development the game is in. If it's a high-budget title, the producer selects a group of testers and makes one team leader. The team leader then has a briefing with the producer to discuss what the priorities are in this stage of the game.

After this meeting, the team leader assigns work to the team. This work largely depends on what kind of game it is. If it's a shooter, usually each tester will be assigned certain levels and will be expected to find any bugs in those levels. In the case of a point-and-click adventure game, all testers play through the game, each one looking for a specific type of bug.

Bugs are generally classified as follows:

A Class: This category is for game-breaking bugs.
• Game freezes.
• Player gets stuck.
• Console hard locks.
• Level cannot be completed.

B Class: These are serious bugs that production must address but do not break the game.
• Missing textures.
• Broken A.I.
• Missing audio.
• Drop in framerate.

C Class: These are minor bugs that are generally fixed in the "polishing" phase of development.
• Texture seams.
• Spelling mistakes.
• Missing credits.

When the game is in its polishing phase, it needs to go to Microsoft/Sony/Nintendo for certification. Certification is basically a list of rules that these companies make that every single game released for their console must follow. For Microsoft it's called TCR (Technical Certification Requirements). Sony calls it TRC (Technical Requirement Checklist). And for Nintendo it's known as LotCheck. These rules are not publicly available, and it's against my nondisclosure agreement to disclose them.

They are basically technical aspects (i.e., does the game display properly in both 16:9 and 4:3 aspect ratios? Can the system play the game for X number of hours straight without locking up, etc...). My specialty was certification. Almost every game listed above was cert-tested by me.


Agatha Christie: Murder on the Orient Express

AH: Is game testing as fun as it sounds?

SS: Game testing is nowhere near as fun as it sounds. When you first get the game, it usually doesn't work. It is likely riddled with A bugs that make it impossible to play. Cut-scenes are usually missing, and voices and textures are often missing. By the time the game is actually playable, you've been through it dozens of times. Also, there are extremely long hours when a game nears its deadline. I have worked 24-hour shifts before. Do you know how frustrating it is to play a game that you've already beaten 100 times for 24 hours straight? It's maddening!

AH: What kind of input did game testers have, and how does this input get into a game? Did you have a clear line of communication with the producers and other higher ups?


SS: Game testers have a significant amount of input. We do have direct communication with the developers through our online bug reports, but in those we just discuss what is broken. If we have a suggestion for a gameplay feature, we have to discuss it with the producer first before mentioning it to the developer. This is because the producer is responsible for managing time and resources involved with the project; therefore, he must approve everything.

AH: Did your work in game testing change how you perceive games you play in your leisure time? Was the amount of leisure time you spent playing video games reduced due to playing so many games in your work time?

SS: Sometimes when I'm playing a game, I'll see something that looks like it could be broken. I found a stuck bug in Halo once, on the first level, no less. I obviously never played any games in my leisure time that I've tested, but I am a fan of Mario Kart and pretty much any real-time strategy game that doesn't involve massive armies of half-naked men. I hate MMORPGs, and I think the people who play them are socially disabled


Land of the Dead: Road to Fiddler's Green

AH: A development studio can't be like a normal work environment. What's the culture like in a development studio?

SS:
It sucks. It's like being in high school. Everybody you work with acts like that asshole kid with the glasses and the rich dad. You know the one -- they probably like Half-Life and have an anime collection. Because of this, nobody in production takes you seriously because of the company you keep. As for the studio itself, it's usually pretty nice. Leather couches, big TVs, fancy kitchen, pool table, etc....

AH: In terms of other games, how aware is a studio like Groove Games of other games out there?

SS:
Depends on how fat and Mormon your producer is. In my case, my producer was very fat and very Mormon and didn't have a sweet clue what was going on anywhere besides the closest greasy spoon. In fact, I suspect that same greasy bastard arranged a company takeover with a foreign game company, which cost us all our jobs. I'd be surprised if he even owned a game console.

AH: The process you described was fairly objective. Did the game testing take into account gameplay elements that were technically sound but needed refinement from a subjective point of view? For example, perhaps the targeting technically worked but didn't quite "feel right." Would you work on something like that, or was it a matter of getting the element to a specification required by the production team?

SS: In the case of your example, it could be logged as a suggestion. The devs check the suggestions. If they agree, a conversation begins involving everyone from testers to CEOs. If they decide to go with the new change, the tester who suggested it will certainly not be credited for it.

AH: Groove Games is not known for their stable of highly regarded games. Was there perception within the company that reflected this outside perception?

SS: Nah. All companies release crappy games. Groove did it pretty much exclusively. It is surprisingly very profitable.

The target consumer is old people. They don't know anything about games. When they want to pick up a birthday gift for little Johnny, they just go to Wal-Mart and look at the boxes. If it's cheap, has guns, and comes in a pretty box, they are sold. Then little Johnny gets to play a delightful game where he gets to murder the leader of Iran for "possessing weapons of mass destruction" (which you find no actual evidence of in the game).

AH: Were there any instances the developer didn't fix known issues?

SS: Yes, all too often. Mostly C Class bugs. Usually, if there are bugs present, it is a spelling mistake or a texture seam.

AH: Why would a publisher put out games with known issues? Was it because time was a factor? Was it a case of building the game for the budget audience?

SS: It's usually because they are too insignificant to delay the game for. If you miss the deadline for Wal-Mart shelf space, it could be another 6 months before you're guaranteed a spot. It's not worth delaying the game for a spelling mistake in the credits or a texture seam on a tree. All games are shipped with at least a few documented bugs.

AH: Did the game-testing side ever feel frustration in terms of identifying issues that weren't going to be fixed, or were the issues something that even your team couldn't get to?

SS: Our job was to simply find the bugs and document them. Personally, I couldn't care less if they got fixed or not. I got paid by the hour. As long as my team documented it properly, it doesn't concern me.


Secrets of the Ark: A Broken Sword Game

AH: What was the difference in terms of testing different genres? I can imagine a first-person shooter with free movement is a different animal than a point-and-click adventure.

SS: I have tested first-person shooters, real-time strategy games, point-and-click adventures, Olympic sports, racing, and role-playing games. All of them have to be approached differently. For example, in a FPS I usually focus on the level geometry: I'll jump into odd places, try to leave the map, or find exploits. In a real-time strategy game, I may make a chart and measure how much damage each unit inflicts on each of the other units to make sure the game is fairly balanced.

AH: Would you have been more concerned with the final quality of the product if the game had a higher cache, say a Halo or Resident Evil?

SS: If the game is triple-A, publishers and developers will spend significantly more on testing. I personally put the same amount of effort into every game.

AH: Could anything motivate you to think of the job as more than a wage? Can game testers even have or develop personal investment in a product?

SS: The only reason I was a tester was because I wanted to become a game artist. A lot of staff at video-game studios started as testers. When you are a tester, you have no say in anything other than what's for lunch.

AH: Knowing what you do now, would you re-enter the video game industry? If so, in what capacity would you want to return to gaming?

SS: I plan to re-enter the video-game industry when I can be my own boss -- not a moment sooner. I have been working on my own game for several years now. When it reaches the point where I have a playable demo, I'm going to start my own company and shop around for a publisher. Until then, I am very happy teaching 3D animation.


Steve Sutton continues to teach in St. John's Newfoundland.

 
Problem? Report this post
ANDREW HISCOCK'S SPONSOR
Comments (10)
Bitmob_photo
February 27, 2010

Just reinforces my decision to never work in QA.  Great interview though.  A really enjoying read.

Default_picture
February 27, 2010

Great Q&A, Andrew. Sutton resonates with me, I personally find it maddening to fully explore open-world games just to feel "I've been everywhere": the thought of having to go everywhere to find bugs—namely trying to leave the map or get stuck—terrifies me.

Default_picture
February 27, 2010
Default_picture
February 27, 2010

My brother did testing for cell phone games a few years ago.  Now he manages a department at that company.  Apparently, the fun is in creating and working with ideas, not just the technical "make sure your controller doesn't blow up" end.  it seems like the only way to have any semblance of fun in the industry is to just make a game yourself with a small group in your free time, shop it around the Net, and hope it gets popular enough that some publisher wants to make a console port.

Default_picture
February 27, 2010

Good interview! It is nice to hear some straight talk about industry professions.  

Default_picture
February 28, 2010

I enjoyed reading that interview. Very enlightening.

Brett_new_profile
February 28, 2010

During my brief time in game development, I worked as a compliance (i.e. certification) tester. In a lot of ways, it's even duller than standard testing -- you've got your list to work from, and if a bug doesn't fall under your list, it doesn't concern you.

We were under a lot of pressure too, because pretty much any certification bug is A-class -- one mistake and Sony/Microsoft/Nintendo kicked the game back to you, wasting valuable time and possibly missing the desired release window.

In other words: It's a lot of responsibility with little reward. I don't recommend it for anyone.

Default_picture
March 01, 2010

Despite all my knowledge of how bad testing is, I did try to get a tester job about six years ago. When I lived in Arizona I applied at Rainbow Studios and D3, and got an interview at D3 but didn't get the job. The closest I got to testing was being chosen for closed beta testing on Sid Meier's Pirates! I know a lot of companies open up beta testing to gamers but I rarely apply. This was a remake for my favorite game so I took it seriously and think I did a good job. All of the beta testers got their names in the credits, sort of. My name's not actually in the printed manual on in the game but it is in a readme file on the disc that no one will ever look at :)

Default_picture
March 01, 2010

@Jonathan: Be glad you didn't get picked up at D3. When I was at THQ in Phoenix some of my co-workers went to work for D3, when their contracts ran up. Apparently it was quite horrible; low pay and no over time. Also one side of the building had the AC go out. This never was fixed. That portion of office would be consistently 100+ Degrees 6-7 months out of the year. 

Default_picture
March 02, 2010

@Nathan: That does sound awful. I ended up at a job in Mesa that had a warehouse with no air conditioning. My job wasn't to actually work in the warehouse but for some reason I often ended up in there. Very hot :)

I do have a friend that was a tester at Westwood Studios many years ago. I'm not sure how accurate this is but he said someone in QA was leaking data and since they couldn't figure out who it was they fired all of the testers.

You must log in to post a comment. Please register if you do not have an account yet.