Tech tricks that make computer games look real

A police officer approaches an abandoned industrial building with graffiti. His body camera, which shakes as he moves forward, captures the scene.

It’s cloudy. Weeds are invading through cracks in the pavement outside. A dog is barking somewhere in the distance, but no one seems to be around.

It became clear that people were hiding here in the dim, rubble-filled interior. people trying to kill him.

The top comment below a YouTube video showing gameplay of Unrecord, an upcoming title by French indie game studio Drama, reads, “Of all the games I’ve seen, the only game I’ve ever seen that made my brain believe it was real. There is a game of “.

The video racked up millions of views in just a few weeks and caused a sensation in the gaming industry. Some commentators on social media questioned whether this was really a game and, if so, whether it was actually a bit too real and a raw experience.

Drama declined to be interviewed by the BBC, saying “we are currently busy dealing with investors and publishers”.

However, the graphics in various games are visibly sophisticated, perhaps approaching what is known as “photorealism”: indistinguishable from real-world photos and videos.

Piers Harding-Rolls, head of games research at Ampere Analytics, says that the Unrecord demo looks so real thanks to some clever techniques.

Harding Rawls points to a shaky camera that mimics real crime scene footage. Dull lighting, gritty ambiance, and the hustle and bustle of the city in the background all help.

But does that make some people uncomfortable?

“The setting is very reminiscent of some of the more terrifying footage we get from real life,” notes Harding-Rawls.

In a statement posted on Twitter, Drama said the game was not inspired by any specific real-life event.

Also, if you look closely at the video still images, you’ll notice that there are some objects and textures that don’t look very realistic. This may not be a problem, but it detracts from the idea that the game is photorealistic.

Harding-Rolls pointed out that graphics advancements in general are important to the gaming industry, saying, “Consumers definitely want that. They look at things and say, ‘Wow, this is great.’ I love to think about it.”

Rachel McDonnell, Professor of Creative Technologies at Trinity College Dublin, also agrees that the Unrecord video is impressive, but points out that certain character animations are a bit awkward.

These remind me of the movements of characters in other games who fall to their death in pre-programmed sequences.

“Animation hasn’t quite caught up with game rendering yet,” she said, adding that making crowds look particularly real.

“You’ll still see them acting very strange, running around in circles, getting stuck, and then you’ll be out of the game very quickly.”

Mark Witten, president of Create Solutions at game software company Unity, says today’s most realistic content relies on highly detailed 3D modeling of objects.

Last year, Unity showed off a computer-generated clip of a lion and its cubs featuring two million individually rendered pelts.

“Otherwise, it doesn’t come across as photorealism,” argues Witten. The company has also developed a highly lifelike model of a human, whose facial expressions are controlled by a digital doll.

He added that there is still room for improvement. There are many other materials that are difficult to simulate, such as clothing, but are still far from looking photorealistic in games.

One of the key new technologies for game graphics is the Neural Radiance Field (NeRF). California-based Luma AI specializes in this and says it already has customers making games using the technology.

NeRF is an artificial intelligence (AI) system that can represent objects and scenes captured in photos and video footage in the real world.

“By viewing these images from different sides, the network learns how light reflects off everything,” explains Luma AI co-founder Amit Jain. “Measure light, learn from light.”

For example, a leather seat on a motorcycle and a headlamp reflect light very differently, making it very difficult to simulate in the game. Nerfs can help automate the process.

Some of the best game graphics today use something called ray tracing. It accurately simulates the way light reflects off surfaces and creates glowing effects around neon signs.

Brian Catalzano, Nvidia’s VP of Applied Deep Learning Research, says AI has made it possible to create these effects in games, despite only modest improvements in chip performance.

“We have to get smarter about how we build the world and how we render it,” he explains.

Ray Tracing: Overdrive, a new mode in the action-adventure game Cyberpunk 2077, shows the difference this makes.

Nvidia says its Deep Learning Super Sampling (DLSS) technology allows developers to create high resolution, high frame rate graphics with ray tracing with the help of AI.

“The model is trained to know what things look like in the real world,” Catalzano explains.

Epic Games VP of Engineering Nick Penwarden says games are sometimes becoming more and more difficult to distinguish from reality.

But he says it’s still very difficult to render certain materials convincingly, such as an iridescent layer of oil on top of a puddle.

“These are aspects that we don’t yet have the ability to simulate in real time,” he says.

And it’s important to do it on game consoles and home PCs. Movies that feature computer-generated imagery use huge computers and can take minutes or more to render individual frames.

The most popular games of the future may not need to be photorealistic. Think Minecraft or Epic’s Fortnite. Both are wildly successful, but far from photorealistic.

But improvements in lighting effects and material simulations will help artists working on games of all kinds, Penwarden argues. It can also give stylized or cartoon-like environments more depth and complexity.

“One of the big advantages of being able to create photorealistic images is that technology can kick off a lot of the work automatically,” he says.