Breakthrough AI will soon be able to generate entire 3D worlds from 2D videos

AI Video & Visuals


In the latest effort in its never-ending artificial intelligence project announced this year, software giant NVIDIA unveiled a program that can create perfect 3D replicas of objects based solely on 2D video footage.

Called Neuralangelo (neural and Michelangelo combined), the software can generate lifelike virtual replicas of complex 3D models of buildings, sculptures, complex structures, and much more.

“The 3D reconstruction capabilities provided by Neuralangelo are of great benefit to creators, helping them recreate the real world in the digital world,” said Ming-Yu Liu, senior director of research and co-author of the Neuralangelo paper. says.

“This tool will finally allow developers to import detailed objects, whether small statues or gigantic buildings, into virtual environments for video games and industrial digital twins.”

NVIDIA isn’t the first company to create such AI models, but it’s probably the most advanced. Previous versions struggled to capture repetitive texture patterns and detailed colors, but with Neuralangelo this is less of an issue.

Using a 2D video of an object, structure, or scene (all taken from different angles), the model selects specific frames and maps the angles of interest for a complete view of the structure .

Once the camera positions for each frame are determined, the program creates a rough 3D interpretation of the scene. Rendering is then optimized and details sharpened to produce a final 3D object that can be put into virtual reality or used in a variety of industries.

Although the program’s functionality has been revealed, the software itself has yet to be released to the public.

NVIDIA is one of many companies making big bets on artificial intelligence this year. Adobe, Google, OpenAI, Microsoft, and many other giants have all poured billions of dollars into creating models that will take over the industry.

We’ve seen AI create music, write poetry, complex chords, and even attempt to create award-winning artwork. But 3D generation is one of the big problems for AI.

Due to the complex and unpredictable nature of 3D forms, they are not as easy to replicate as 2D images or text. OpenAI has tried to figure this out with their Point-E project, but admits it’s a complex project to create.

If NVIDIA and others that follow suit can finally create an AI 3D model generator, it will have far-reaching implications for the world of artificial intelligence. One of the biggest problems AI art has faced so far is the inability to understand complex shapes.

Trained through 2D art examples, these generators struggle to understand hands and complex shapes. Including a 3D generative model will give you a better understanding of the model you are trying to replicate.

read more:



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *