The future of videogame AI

Videogames have journeyed from pixel sprites to near-photorealism, but our expectations for how characters should look have been satisfied more easily than our desires for how they should behave. The next generation of technology may be able to render the way light glistens on the meniscus of an eye, or scatters beneath soft skin, but will the owner of said skin be able to find her way out of a room without hitting herself with the door?

We’re not placing bets yet. Historically, even our most basic hopes for convincing AI have been left unfulfilled: squadmates should ideally try to avoid spike traps rather than bumble into them repeatedly, for instance, or take cover on the side of scenery that isn’t being riddled with gunfire. The real battleground for realistic behaviour will not be fought with bullets, however, but words.

Skyrim, LA Noire, Mass Effect and Heavy Rain are among the handful of recent games that place emphasis on the complexity of human interaction and the importance of empathy, but whose restrictive dialogue trees, cutscenes and canned character barks struggle to match the expectations set by the fidelity of their worlds. The glittering eyes and sharply defined pores of the future will only raise the bar further: we’ll want unique characters in their hundreds who don’t repeat responses, whose relationships are a palimpsest of previous interactions, whose conversation is peppered with idiosyncrasy, and who can respond to a constellation of possible prompts with the same nuance, dynamism and integrity as a real person. Wouldn’t that be truly next gen?

“These sorts of things aren’t represented in the model for even the most dialogue-heavy mainstream games,” says Emily Short, a game designer renowned for psychologically complex interactive fiction. “You look at BioWare’s large-scale RPGs, games with cutscenes, and either you’ve got a situation where something has been pre-recorded to be played through one way, or prefabricated dialogue trees. There’s not an expressiveness to the way the dialogue tree mechanic works: you can only pick a choice.”

This need not be so. Short herself has worked with former Sims 3 programmer Richard Evans to set up experimental game studio LittleTextPeople, established to investigate the potential of dynamic, richly social AI. Its first game, Cotillion, has proved promising enough for Second Life developer Linden Labs to acquire the company. The game’s a sort of Jane Austen novel simulator (‘cotillion’ is a dance from the era), a comedy of manners in which you pull the strings. Complex characters interact with each other while pursuing their own motivations, and the game narrates the subsequent entanglements via text, with paragraphs of description and dialogue generated in realtime, and subject to your manipulation.

“Our project isn’t so freeform that you can type in anything, but it’s more fluid [than dialogue trees],” says Short. “The simulator Richard created has the potential to have people exhibit finely grained eccentricities, so it’s possible to write characters who grunt when spoken to, or like to talk about themselves and take the conversational initiative more than they should.”

During a talk at GDC 2012, Short outlined some examples: “There’s a scene where the characters have been in a carriage accident in the middle of the night, and they’ve been left at the roadside by the driver of the carriage, who knows this neighbourhood is haunted. So he’s freaked out and left. Elizabeth’s just got out of the wreck, but she didn’t see the driver leave. So the world model has given her the question: ‘Where did the driver go?’ Lucy knows the answer to this question, so she says, ‘Well, the driver thinks this area has a bad reputation, so he fled.’ Her particular phrasing of that is customised. Her character is supposed to be a little bit diffident, so she has her own way of saying that. Then there’s a standardised response. The doctor says, ‘Actually, I bet he left because he was scared of us being mad that he wrecked our carriage.’ And that’s tagged as potentially meant to be humorous. And Lucy has the opportunity to decide: ‘Do I find that humorous or not?’ She does, so she laughs. This model gives us the opportunity to have very fine-grained characterisation, and to have characterisation where people express personal eccentricities through the style and the shape of their conversation.”

Ben Sunshine-Hill (left) works for Havok programming AI, while Mike Treanor is a PhD student, and one of the designers of Prom Week

By Short’s own admission, Cotillion is more like acting or improvisation than playing a game. And being text-based, it remains at a distance from the current mainstream vogue. Part of that is simply budgetary (“Text costs nothing,” says Short), but it is also by design: “The level of nuance to the performance of an action, the interiority of the character’s emotions, and the possibility of having conflicting motives were prohibitively difficult to represent visually.”

Short and Evans aren’t the only explorers in this field, though – the University Of California in Santa Cruz has set up the Expressive Intelligence Studio (EIS) to cover similar ground. It made waves with 2005’s Façade, a somewhat wonky social simulation in which you navigate a tense evening engagement with a fractious couple. It was an ambitious project whose potential was apparent, but faltered in the naturalism of its execution.

“Façade is trying to do so many things at once,” comments Short. “It deals with three hard problems: how you process natural language input, how you represent character emotions, and then how you do AI-based drama management on top of that, and make sure the interaction leads to some sort of increasing tension, crisis and resolution. Each of those three things is so difficult, [and] there’s so much more to be done on them individually before we get a clear idea of how powerful they will be when put together.”

Nonetheless, Façade’s groundwork has been built upon by the studio’s subsequent research projects, and the latest, Prom Week, is a social engineering game in which you navigate the flaring tempers and fraught relationships of an American school. It’s been described as “the Crayon Physics of AI”, a game in which the entire web of social interrelations is laid before the player.

“Prom Week has a very sophisticated model of what it means to be social,” explains Mike Treanor, a PhD student, and one of the designers behind the game. “Actions aren’t arbitrary, they make up a theory of social interaction, and to play Prom Week is to poke at that theory and try to understand it. Every interaction is taken in order to change the world state with someone else: I just complimented you to make you like me a little more. And that is the interaction in Prom Week – you choose which social exchanges characters make with one another.”

It’s Machiavellian in the extreme. In order to get the school Linux geek to become prom king, one of his many objectives, you have to rig the jury. You do this by building a friendship with the head of the prom royalty selection committee, which in turn requires you to bully a mutual enemy. And although each mission and its goals are related to a specific character, you can select any character onscreen and command them. Clicking on one character and then another gives you a set of possible interactions depending on the current relationship between the pair.

“Dialogue isn’t authored for one character to say – any character could say [the words] if the situation’s appropriate,” says Treanor. “But within the lines of dialogue themselves, there are points where characters insert the things that reflect who they are and their personal social context. So for 6,000 lines of dialogue, you get what feels like 18 characters with unique things to say.”