Gaming needs photorealism to properly convey emotions

Default_picture
Monday, August 20, 2012
EDITOR'S NOTEfrom Jason Lomberg

Gaming as a storytelling medium is artificially handicapped due to technological limitations. Whereas actors in film can express complex emotions with the flick of an eye or a smirk, character models in gaming are a tad less...expressive.

Heavy Rain

During an interview with GamesIndustry International, 2K Games boss Christoph Hartmann made the statement that certain types of games are unable to achieve particular emotions due to the limitations of current technology.

He surmised that if we were able to achieve photorealism in games, then we could properly convey those emotions.

"Recreating a Mission Impossible experience in gaming is easy; recreating emotions in Brokeback Mountain is going to be tough or at least very sensitive in this country...it will be very hard to create very deep emotions like sadness or love, things that drive the movies. Until games are photorealistic, it'll be very hard to open up to new genres. We can really only focus on action and shooter titles; those are suitable for consoles now. To dramatically change the industry to where we can insert a whole range of emotions, I feel it will only happen when we reach the point that games are photorealistic; then we will have reached an endpoint and that might be the final console."

Naturally, this was met with a fury of gamers coming out in droves to declare that, "GAMES DON'T NEED GRAPHICS!!!!" and other arguments that fail to really understand what, I think, Hartmann was getting at.

 

In this industry, we love to compare video games to movies. For many reasons, I believe it's a poor comparison, but for the sake of the argument we'll start there.

In film, the actor can display a wide range of emotions without ever having to spell out exactly what they are feeling. You could place an actor in a white room with no sound and no dialog whatsoever, and he could convey many emotions and probably even tell an entire story. In movies like Brokeback Mountain and many other films, there are parts that feature no dialog and rely solely on the actors to move the audience. This is, however, something games struggle with.

Let's compare a scene from a movie with no dialog and real characters with a similar clip from a game.

The first is a scene from the movie Before Sunrise. It perfectly captures that moment when you first fall for someone and does so without the use of any dialog. You could even mute the audio so you don't hear the music and you'd fully understand what the characters are feeling for each other.

Then there's this from the game Heavy Rain:

Boy, you guys sure are right! Better, more detailed, more realistic character models would never help that completely not-at-all-awkward scene -- stupid quick-time events notwithstanding. Yeah, I understood what was happening and what they were trying to convey, but that doesn't mean that it made me feel anything.

Games are the amalgamation of all other art forms. They have the ability to tell stories through a beautiful mixture of drawings, level design, music, storytelling, character development, and player interaction with the world. This is something that no other medium can accomplish. However, gaming has not reached a level of graphical fidelity that it can properly convey emotions without the aid of another art form such as writing or music.

Let's look at some recent games that go for human-like character designs. Games such as Heavy Rain, Mass Effect, Grand Theft Auto IV, and Red Dead Redemption, just to name a few. All these games try their hand at varying levels of emotions and exceed to some extent, but if we remove the actual dialog and music of those games and put the onus of conveying a feeling solely on the character model, can they do it? I don't think so. 

Heavy Rain's awkward love scenes look like two clay models being rubbed together by a little kid. Mass Effect's weird repetitive motions, blank stares, and Shepard's creepy smile during dialog look and feel fake. Grand Theft Auto IV and Red Dead Redemption both share similar design elements, and I couldn't imagine a scene from either game playing out without the help of dialog.

Do all these games have great writing, though? Well, some of them do, and that's the key. That is the only thing that makes these characters feel like they are truly conveying emotions to us.

Thomas was Alone

Is that a bad thing? Certainly not. But I believe Hartmann has been taken horribly out of context when speaking of "photorealism in gaming." It makes little sense that he would ever make that point with games like Thomas Was Alone, Shadow of the Colossus, or The Maw in mind. Clearly, he would be talking about games that prominently feature humans doing human things in human-like environments.

What boggles my mind is that so many people have become so vehemently argumentative about anyone ever suggesting that graphics could (or should) be better that they have completely failed to understand, what I think, are the intentions of his quote.

You can't have photorealism of something that isn't real. A game like The Maw is able to convey emotions without the use of dialog or photorealism. Then again, there's no such thing as photorealism in The Maw because nothing in The Maw is real.

When a game can make me feel what I felt while watching that scene from Before Sunrise without the aid of dialog, then I'll change my tune. But until then, I welcome all advancements in graphics, because creating emotions is more than just dialog and music, and right now, that's all games have.

 
Problem? Report this post
BITMOB'S SPONSOR
Adsense-placeholder
Comments (13)
100media_imag0065
August 20, 2012

Play Ico. Not a word spoken between the two that we understand, and their relationship is a thing of legends. Same goes for Shadow of the Colossus. Spoiler Alert: I know people who cried like babies when your horse falls off a cliff, because that relationship between the two of them was so powerful. I get where you are coming from here, but I don't agree with it. If anything, realistic graphics and realistic facial animations make it harder for us as gamers to really associate with the characters. The less photo realistic it is, the better we accept it as gamers.

However, we have been known to fall in love with character simple because of their animation, facial especially, and their voice actors. Take Alyx from Half Life 2. Now, I don't know a single gamer who doesn't love that character. And most importantly, she conveys all of her emotions through her face. You can clearly see when she is happy, upset, and yes, in love with Freeman. There are quite a few spots in HL2 and the Episodes where she looks at you, and, without saying a word, she says everything you need to know with just her eyes. Yet, at the same time, she isn't photo realistic. Far from it, it is actually a very dated game.

Yet, when you go back to it, even to this day, there isn't another character in video games that does a better job making you love and care for them. And Valve accomplished it all without photo realism. All you need is the right voice actor, and the right animators. Graphics have nothing to do with it. Some of the most powerful moments of Half Life 2 are when no dialogue is spoken, and all you have to go on is the characters faces.

Default_picture
August 20, 2012

I have played ICO and Shadow of the Colussus.

The part that I may not have properly explained in my article is that I don't believe games will ever actually look like real life, but when I'm talking about photorealism, I'm talking about greater detail in character's faces and better animation. Like Pixar. Pixar movies don't look like real life, but they can mimic at very great detail the expressions and manorisms of humans very well.

Sure, I didn't NEED that for my relationship between ICO and Yorda, but that's because there were other factors that pushed my emotions in that direction. But there couldn't been a scene between ICO and Yorda with no dialog(not that there was much to begin with), no music, no special camara angles, or anything but the facial expressions between the two that could really create the same real, human emotion that two people can feel between each other. 

And with SotC, I agree, but that was between a man and a horse. I'm talking about humans and humans. But I still see your point on both accounts.

As far as Alyx Vance, you basically proved my point with your statement. You didn't always need dialog from her to understand her emotions. Yeah, she wasn't perfectly animated, but Valve was going for the most realistic animations and character detail that they could. The only thing stopping them at the time was the technology.

You're lying to yourself if you think Vavle won't try to keep pushing their characters to animate more realistically and look more realistically in that games that feature real, human characters.

My point isn't that ALL games need photorealism. I don't believe that at all. But a more detailed, better animated Alyx Vance would only improve Valve's storytelling. And I'm sure we'll see that in the next Half-Life game.

Which, the re-titling of the aritcle doesn't help because I don't believe that all games need photorealism to properly convey emotion. My original title was simply "In Defense of Photorealism in Games." 

http://pixelperfectmag.com/in-defense-of-photorealism-in-games/

100media_imag0065
August 21, 2012

So you agree with me then? Valve, using dated technology that isn't even close to photo realism, was able to convey a world of emotions with simply a characters facial animation. I think that proves that realism has nothing to do with it. Neither does a voice actor when you have the right animators.

Any talented animator can make you care for a Banana if they wanted to. Check ou the Bioshock Infinite gameplay demo. When she grabs your hand and begs you to kill her if she is captured. Mute the video and just watch her face. What you have there is a very cartoony game, conveying a VERY powerful, desperate emotion of fear.

Again, the only thing you need is good animators. That is it. Nothing else. You're crazy if you think anything else is needed. Take any number of the talkented animators in this industry, and ask them to make a silent game filled with emotion. I guarantee you they would knock it out of the park. I guarantee you they can tell an entire story without saying a word, using only the characters facial animation.

You don't need actors in front of a camera. All you need is the talent tha knows what they are doing.

Default_picture
August 21, 2012

Ed, I think you may not understand the intent of this article. Or maybe I'm just taking Jared's reasoning a logical step forward.

Yes, talented designers can create an emotional attachment to digital objects regardless of how sophisticated the detail and animation is. A perfect example is the Weighted Companion Cube from the Portal series. One could almost consider "it" a meta joke since GLaDOS makes specific reference to the Cube as an inanimate object yet proceeds to make you care for it and then incinerate it.

But then, the Weighted Companion Cube is a non-human object. And certainly, we've experienced vicarious emotion from human-like characters in gaming that don't have photorealistic features.

But a lot of this emotion can be derived from external factors -- the music, script, story, atmosphere, etc. The digital creation, by and large, can't emit subtle emotions in the same way a real-life actor can.

And it would be a great deal easier for said designers if the digital characters could emote in such a way. Imagine what a talented designer could do with such a creation. Take two sets of tools -- one with photorealistic capabilities and one without -- and add a talented designer. All things being equal, I believe that photorealism conveys a huge advantage.

Default_picture
August 21, 2012

 

Alyx works, somtimes yes, but she doesn't display a wide range of emotions without dialog. The few times she does it's very, very simple things like a smile. And let's be honest, a smile at Gordon doesn't qualify as amazing animation. A smile is very easy to convey.

There aren't many scenes where Alyx says nothing but let's her body language and facial expressions talk for her. And Half-Life 2 and it's subsequent episodes are some of my favorite games of all time, so I'm not just arguing for argue sake here.

And yeah, you're example about Bioshock is true, but that's not what I'm talking about in my article. I'm specifically only talking about real, human-like characters. Not cartoony versions of them.

Like Jason said, if we had the ability to animate these characters in a life-like way, it would be an incredible leep in the ability to convey emotions and tell stories in a game. Sure, some companies have done a decent job with the current technology, but there's almost always some external factor that has to exist to help the characters convey their emotions.

Default_picture
August 20, 2012

Nicely written article - thanks for taking the time to put this out there.  I wonder how this same concept would impact violence?  I have no issues running around in GTA doing all kinds of horrible things... but if that random dude I just shot looks close to truly human... hmmmm.  I know LA Noire messed with me a little bit in that way... I'm not sure if it was the graphics concerning the victims, the situations, etc., but that game got to me in a way different from any other game.  Keep writing! 

Default_picture
August 21, 2012

Thanks!

Yeah, I'm certainly not saying all games need photorealistic graphics, because it would be uncomfortable actually seeing the pain on a character's face in GTA when you are runing around doing horrible things, but some games could definitely benefit from it.

Default_picture
August 20, 2012

Animation (Pixar!) is the death knell for any argument claiming that photorealism is required to convey emotion. This isn't to say we shouldn't try for better, more realistic graphics, but graphics aren't a magic pill that will suddenly make all our concerns disappear. I would much rather have a well written, well voice acted game with excellent mechanics that looks like Final Fantasy VII than a photorealistic game with nothing else good to say about it.

Default_picture
August 20, 2012

Sam, you're comparing two extremes, but it's rarely an extraordinary, non-photorealistic game like Final Fantasy vs. a pretty title with nothing under the hood. Usually, it's somewhere in the middle. And, in any case, this misses the point.

Final Fantasy VII plays all sorts of tricks to express complex emotions -- a moody score, the (unspoken) dialogue, the writing, and the atmosphere. And this is largely due to contemporary technological limitations. If the designers at Square could've manipulated their character models to emote with subtle facial expressions, the experience would've been much improved.

I don't want to be reminded that I'm playing a video game. I want to get lost in the story and "acting", such as it is. And nothing breaks the suspension of disbelief more readily than blocky character models and blank, expressionless faces.

Default_picture
August 21, 2012

"All these games try their hand at varying levels of emotions and exceed to some extent, but if we remove the actual dialog and music of those games and put the onus of conveying a feeling solely on the character model, can they do it? I don't think so."

But why should they? Why would anyone want to handicap themselves like that? Just to say that "Graphics aren't realistic enough"?

Graphical fidelity has become a crutch in the gaming industry. Developers will make a high fidelity game, but then when something breaks the immersion they say "the graphics aren't good enough".

No.

It's usually other problems that cause the break in immersion such as poor animation (remember photorealism means it will look realistic in a still frame), stilted voice acting, bad music, badly concieved mechanics, I could go on. Often times it's anything but the graphical fidelity causing the break in immersion but it's the only thing that get's looked at and that's the real problem with the industry.

Let's look at that Heavy Rain example again. You mean to tell me that the clumsy animation, the quick time events (widley regarded as one of the worst mechanics in gaming to date) and bad audio sync are going to magically be fixed by having more detailed graphics? Don't think so.

When a character like Sora from Kingdom Hearts can convey a wide range of emotions without so much as a line spoken but someone like Shepard can't, it's not a problem with graphical fidelity, it's a problem with other aspects. When the industry at large stops using graphical fidelity as a crutch and starts actually fixing other problems BEFORE graphics then it will produce more realistic emotions.

Default_picture
August 21, 2012

I feel like you didn't really read what I wrote. I am in no way saying that if graphics were better, then all problems are fixed. You're going into this with your mind made up before even hearing me out.

Anyway, I'm not saying that anyone would remove all the extra components of a game just to prove a point that graphics need to be better. My point was that without the aid of something else -- music, dialog, etc. -- games can't stand have a scene with just the characters facial expressions and body language communicating to each other. And I think that's a problem.

"Graphical fidelity has become a crutch in the gaming industry." Can you provide any examples? Because I don't feel that way at all. Name a a few games that are completely devoid of anything good but look amazing. I'm sure there's one or two examples, but even the companies that seem to care too much about graphics (i.e. Crytek) still put out very solid games.

What you're missing, or maybe I didn't convey it well enough, is that I'm not saying "if graphics were better then everything would be solved." But if you already have great writing, great voice acting, great music, etc., then having better graphics (when dealing with real life characters) isn't a bad thing at all and would greatly increase the story and character development and developers would be able to achieve certain scenes between characters that they could never do before without the aid of dialog.

When it comes to that Heavy Rain example, replace that with ANY love scene in a game. It doesn't work. I was going to use a Mass Effect clip of a love scene as well, but I figured people would understand that I'm not saying that Heavy Rain would have been a great game if it just had better visuals. I was saying that if you take a love scene with real actors versus a love scene from a game, with the current technology, love scenes in games are completely terrible. That's the only point I was making.

A character like Sora isn't real though. Kingdom Hearts is going for a cartoon look, and that's not at all what I'm even talking about. You're completely missing my point. I'm talking strictly about real human characters. I had a line about that at the beginning of my article, but it was edited out. Check my original to see: http://pixelperfectmag.com/in-defense-of-photorealism-in-games/.

"When the industry at large stops using graphical fidelity as a crutch and starts actually fixing other problems BEFORE graphics then it will produce more realistic emotions." Again, I feel like this is just an argument that some people bring up because they hate how popular CoD, Halo, or Gears are. How often do any of these developers talk about graphics? Rarely. Crytek is really the only company that I've heard talk a lot about graphics, and their games aren't terrible at all.

Saying that the industry cares too much about graphics is just an excuse that I think a lot of gamers like to spit out because their favorite games are underrated. 

Default_picture
August 21, 2012

You're right, I am going into this with my mind made up already because I have this same argument before. Hartmann isn't the first. Lead developers of Epic Games, Id, Crytek, Infinity Ward, EA, etc have all pushed for graphical fidelity over anything else becaue they think that's the most important thing.

As for the games, Look up CoD, Battlefield, Crysis, Ghost Recon, etc. And for me, "good" doesn't mean visually stunning and it works (barely in some cases), but that it's an all around package. The examples I gave are all visually stunning but ultimately shallow and it's not the fault of graphical fidelity.

Yes I did read what you wrote. Both in the article and in your reply. You are saying that if everything else is exceptional, then only graphical fidelity is going to break the immersion. I'M saying that developers think that's already the case when it's nowhere near it and getting those up to speed is FAR more important than how it looks in a still picture.

As for Sora, my point on that is that a "non-real" character can convey better emotions than a supposed "real" character is not a problem of fidelity, it's a problem of the writers not knowing how to convey emotions in the first place and more graphical power isn't changing that.

Now for the last two paragraphs, those I;m not even going to dignify with a response as all those are is insulting people who disagree with you.

Default_picture
August 21, 2012

I won't disagree that games like CoD, Ghost Recon, and the others you named are going for the best possible graphics they can push out, but I don't in any way think that makes them bad games. Do they have deep, thought-provoking or moving stories? No. Do they need that to be a good game? No. But that's just an opinion. Regardless, those games still have really solid gameplay. People don't keep coming back to CoD because they think it looks good, they come back because the gameplay is very tight and very fluid. I'm not even a fan of any of those games, but they have good gameplay, I'm not going to deny that.

I won't disagree with that you wrote in your third paragraph, because most of it is true, but saying that those things are "FAR more important than how it looks" is still an opinion. And that's too much of a blanket statement to say that across the board that graphics should be the least of their worries, no matter what game it is. Not all games are the same, and there are plenty of games with realistic human-like characters that would have benefited from photorealistic graphics.

I understood your point with Sora, but that comes back to my point that with the graphical limitations on games with real, human-like characters, is a problem. But in a game with cartoony graphics, it's a lot easier to convery emotions without dialog. Just look at Link in Wind Waker versus every other Zelda game. So the problem lies with the technology, not the writing, when it comes to realistic characters.

And my point with the last two paragraphs wasn't an attempt to insult anyone who disagrees with me, so I appoligize if it came off that way. My point is that, from my experience, the arguement that developers only care about graphics are always coming from people who love games that they feel are underrated by the larger gaming population.

What I don't understand is why is this even an issue? Companies like Epic, id, Crytek, and Infinity Ward will continue to do things the way they always have regardless of the gaming climate. That's just the kind of developers they are. But there are more developers who don't even get into the discussion about whether graphics matter or not, they just keep making their games. And just because one guy or a handful of companies say "graphics matter," that in no way will ever effect the development of games coming from people like Team ICO or Jonanthan Blow or many, many others. So, I get a little annoyed when people make a bigger deal out of this graphics discussion than needs to be.

If John Carmack were to say that graphics are the only thing that mattered in video games, would Team ICO suddenly stop what they're doing and focus solely on new technology? No, that would never happen. But the people who are fans of those games (which I am very much a fan) are the ones who become so incredibly upset about even the suggestion that graphics matter and then spend so much energy on arguing that that is the only thing the industry is focused on. Which couldn't be further from the truth.

You must log in to post a comment. Please register if you do not have an account yet.