My favorite news story of 2023 had nothing to do with AI. I didn't think it had anything to do with AI, at least until I tried his two demos by the developer that use part of Nvidia's “Digital Human Technology Suite” to “bring game characters” . In life. ” What I'm talking about is this: How deep was the simulation in Deus Ex? A literal piece of paper could set off a laser trap. This is about a 24-year-old video game. Specifically, it's about a very obscure and improbable event in the video game where someone sees it for the first time decades later.
By today's standards, Deus Ex is crude in many ways: small environments, chunky graphics, and clunky controls. But it's also a triumph of design, with every little cog inside it made to work with his six other parts in intentional and unintentional and logical ways.Was he the designer of Deus Ex? plan So that piece of paper could set off a laser trap and blow someone up? No, but they've built a world where even that unlikely interaction plays out in a “realistic” way. We gush about Deus Ex and other immersive sims because, more than most video games, they go to great lengths to ask the question, “Will this work?” logical, meaningful, interesting answer.
Great immersive sims are still impressive decades after they were created, and at last month's Game Developers Conference they announced that they were powered by Inworld AI, which bills itself as “the most advanced AI engine for games.” When I played the demo, that feeling became even stronger. The demo exaggerated the endless wow factor of what could actually be done. talk Talk to NPCs — When I spoke into the microphone, the game processed my voice much like Siri on an iPhone before the NPCs could talk back. That's a neat trick. Impressive, even!
But after the demo, I had a hard time understanding that this technology was more than just a gimmick, and that the wide-open possibility space of “say whatever you want” could lead to meaningful interactions. I was skeptical that it would convert. Because developers still need to plan how to realize what they are given. The dialogue relates to the rest of the game. The more I think about them, the more I find it hard to believe that vast resources are dedicated to a highly reactive world in pursuit of the wrong kind of authenticity, a “natural” but ultimately empty conversation. It felt like a lot of money and effort was being spent. Like Deus Ex instead.
NPC vs. World
Inworld AI uses large-scale language models to enable NPCs to react to interactions and spit out their own unscripted responses. Some of his underlying Nvidia technology handles speech-to-text processing and facial animation. Their voices were provided by actors who had recorded enough lines for the AI to simulate synthetic voices, and their performance was honestly impressive. I've heard worse expressions in games.
In a demo called Covert Protocol, you walk around a shiny (RTX-on!) hotel lobby in an adventure game-like atmosphere, picking up clues and some key points to help you get to different floors and do things. I was looking for an item. Spy stuff. I was able to speak to the desk clerk and a surly man named Diego who was preparing to speak at the conference. Conceiving Diego or engaging him was the key to getting the information I needed to progress. Rather than choosing from a variety of pre-written dialogue options, I was able to say whatever I wanted, so I immediately started poking holes in the simulation. I asked Diego what his favorite Taylor Swift album was. He clearly understood the gist of the question, but instead of mentioning Taylor's name or a specific album, he said he didn't have a favorite, then gruffly reminded me that he was preparing a speech. I was allowed to.
There's potential for cleverness here, and lying about your identity, requesting information in different ways, and interacting with a real voice creates an immersion you don't usually feel when choosing conversation options. The few seconds of visible processing delay actually reduced how “natural” each interaction felt. It was also hard to shake the feeling that I was talking to an empty vessel there for me to explore, rather than a character created by a human writer who had something to say. However, I could see the writers creating a lot of set lines for her NPCs and using this kind of her AI to “fill in the gaps” with additional responses that weren't planned.
What's even harder for me to imagine is how interacting with one of these NPCs would improve most video games. In this buzzword-heavy promotional video for Covert Protocol, Inworld's CEO says his platform “unlocks breakthrough game mechanics” that “dynamically adapt to the decisions and actions NPCs make in-game. “Every player will have a unique experience with Covert Protocol.” . ” To these points I say: One, don't forget Seaman, and two, how many of those “unique” experiences actually matter?
In Covert Protocol, the things I said in my conversations with Diego had to boil down to some very limited results. I was able to get a little information from him or have him talk to the clerk and ask him anything that would be helpful. I progress. The second demo, created by Ubisoft but running on the same Inworld AI technology, had me chatting with an NPC to plan a heist, and she told me how long it would take to complete my objective. Most of my ideas were killed in painfully slow exchanges. Specific suggestions she wanted to hear. It was frustrating, and of course the demo is no indication of how good this technology will one day be. But think about it. Of course, if Ubisoft hadn't already built Zeppelin and parachuting sequences into the game, there's no way the AI would take my suggestion seriously if I said, “Let's parachute from a Zeppelin onto a rooftop.” . Any freedom of conversation should be narrowed down to a chosen course of action or solution.
That's unpleasant. It highlights the fact that you can say whatever you want, but you can't actually say it. do whatever you want. Even if the NPC says something surprising to me, my player agency still remains limited. Sure, human-written dialogue inevitably gives you fewer choices, but as a player I know that all of those choices matter in some way. Particularly in immersive sims, we know that the interaction options we can choose from are part of the limitations of the game world, but within those limitations, everything from intimidate checks to stackable crates has the potential to be used at our disposal. It's a great tool.
Having clearly defined limits rather than illusory freedoms makes the world feel more “real” in itself, or at least coherent.
The essence of a good immersive simulation is that the designer creates a variety of interesting approaches to the situation for the player, using certain explicit levers that can be manipulated, but not in a truly There are also new elements that allow creative players to solve problems. Problems in ways not explicitly designed by the developer. “Thinking outside the box” allows you to discover unexpected solutions. But it's hard to imagine using this kind of her AI dialogue system when developers need to cherry-pick possible outcomes.
So what do the creators of Deus Ex think?
I took this skepticism to Warren Spector, who happens to be the director of Deus Ex. His own upcoming immersive sim (However, none of this AI technology is used). In fact, he's more optimistic about this than I am, although he cautioned that he's far from an expert in all aspects of AI. “Since 2008, I've been complaining about how terrible the conversation around the game has been,” Spector told me in an email. And that hasn't changed much since the 1990s, and it remains the same today. He pointed out that most games still use branching dialogue trees to represent responsiveness.
He said, “The lack of progress in dialogue with NPCs can only be described as pathetic (Holding fair dialogue is a difficult problem, and I myself don't know how to solve it. )” he said. “But branching trees turn conversations into puzzles. They're boring. And it's time-consuming and expensive to generate enough dialogue to keep players satisfied. In 2008, I discovered that true I said we need natural language processing and advanced AI to make progress. The fact is, I'd be a hypocrite if I said I wasn't at least interested! This means that the AI that does this should only be trained based on game-related information. An NPC in a fantasy game talking about Taylor Swift.”
(To be fair to Inworld here, the company's AI technology includes a feature called the fourth wall, which means that no matter how often the player tries to trip up an AI NPC, The AI character is designed to always say a response that is appropriate to the world.''In fact, I suspect there will be a lot of bland bias, like the one Diego told me about his favorite album. doubt).
My pessimistic view is that developers have to consider allowing players to say anything For the NPCs, these free conversations would ultimately lead to unfortunately limited results. Spector doesn't take that for granted, but he also doesn't think AI will automatically open the door to more player agency.
“I'm not saying that AI in itself will limit new possibilities. Who knows what impact a freer approach to conversation and overall behavior will have?” he said. “Anyone who says they know is much smarter (or delusional…) than I am.
“Player agency is important, and there are many ways to achieve it. If you're crazy, you can script a lot of player options. If you're not so crazy, you can script a lot of player options. , allows you to build game systems that simulate physical forces and allow players to do the following: “Taking advantage of the system's possibilities, the tools, the world itself, and their own resourcefulness, Deal with game challenges. The question is whether there is anything else that can be simulated that goes beyond physical forces. It requires great imagination and design work. I know how. AI might be able to help you with that (keep in mind, I'm not an expert on this, so those who are will probably chuckle at my idea!).
I can understand Spector's frustration that video game dialogue hasn't changed much since the '90s, but I actually appreciate the fact that dialogue conveys human emotion while also acting as a type of puzzle. I like it. Immersive sims are impressive in how much you can do within their limits, and those limits are very important. Just as a larger open world isn't necessarily better, many empty interactions end up being better than a few tightly designed meaningful interactions once the novelty wears off. It doesn't mean it's better.
NPC AI technology like Inworld could definitely be improved and used in interesting ways in experimental games (possibly next-gen versions of Facade). But the demo I played only managed to convince me that the way to make a game feel properly “real” is to always follow the teachings of immersive sims. Generative AI doesn't make the game any more fun, so create lots of amazing possibilities by hand.
