Neuralangelo is the latest artificial intelligence (AI) from NVIDIA Research that transforms simple 2D iPhone video clips into detailed 3D structures that virtually perfectly replicate buildings, sculptures, and other real-world objects and spaces. can do.
The platform’s name pays tribute to Michelangelo’s ability to sculpt lifelike expressions from blocks of marble. Similarly, NVIDIA Research says Neuralangelo generates his 3D structures with intricate details and textures from simple video clips.
These 3D objects and spaces can be imported into design applications for editing for art, video game development, robotics, and creation of digital twins.
The concept behind Neuralangelo is so similar to other projects coming out of NVIDIA Research that it might look familiar. For example, last year, NVIDIA researchers created a new tool that easily converts photos into 3D objects. The company, called NVIDIA 3D MoMa, has announced that it will enable architects, designers, concept artists and game developers to quickly import objects into graphics engines for digital manipulation.
Although similar, Neuralangelo builds on that concept, allowing you to import much larger and more detailed spaces and objects. What makes Neuralangelo special is its ability to accurately capture repetitive texture patterns, uniform colors, and strong color variations. These are areas where previous AI models struggled.
According to NVIDIA, Neuralangelo employs instant neural graphics primitives, the technology behind NVIDIA Instant NeRF, to capture these details.
This tool is powerful enough to transform large space drone videos into gigantic 3D objects.
According to the company, Neuralangelo uses videos of objects and scenes taken from different angles and selects a number of frames that capture different perspectives. Next, determine the position of the camera within each frame and create a rough 3D representation of the scene. The model is then optimized for rendering to enhance detail before producing the final 3D object.
“The 3D reconstruction capabilities provided by Neuralangelo are of great benefit to creators, helping them recreate the real world in the digital world,” says Ming, senior director of research and co-author of the paper describing the research. – said Yu Liu.
“This tool will finally allow developers to import detailed objects, whether small statues or gigantic buildings, into virtual environments for video games and industrial digital twins.”
According to NVIDIA, Neuralangelo’s ability to transform textures for complex materials, including objects such as roof shingles, glass panes, and smooth marble, from 2D video to 3D assets far surpasses traditional methods.
“The high fidelity facilitates 3D reconstructions that allow developers and creative professionals to use smartphone-captured footage to rapidly create virtual objects that can be used in their projects,” the company said. I’m here.
Image credit: Nvidia
