US-based GPU maker Nvidia has published a research paper outlining a new artificial intelligence (AI) model that can transform 2D video clips into detailed 3D models.
Research has shown that current generative AI models struggle to capture object details such as repetitive texture patterns, uniform colors, and strong color variations. A new AI technology from Nvidia called Neuralangelo is said to overcome these problems. Neuralangelo provides high fidelity models and the ability to accurately transform textures of complex materials from 2D video.
How will Nvidia AI develop as a 3D printing tool?
Nvidia highlights the potential benefits, especially for creative professionals, who can quickly and easily create virtual objects using footage captured from their smartphones. Potential applications for Neuralangelo are outlined in art, virtual reality, video game development, robotics, and industrial digital twin manufacturing.
The company has already demonstrated the wide range of capabilities of its new AI technology, including recreating Michelangelo’s David and flatbed trucks. Neuralangelo can also reconstruct large environments. Nvidia has successfully created a 3D model of him for a park on its Bay Area campus using bird’s-eye video footage captured by a drone.
“The 3D reconstruction capabilities provided by Neuralangelo will help us recreate the real world in the digital world, which will be of great benefit to creators,” said Ming-Yu, senior director of research at Nvidia and co-author of the research paper. Liu commented. “This tool will finally allow developers to import detailed objects, whether small statues or gigantic buildings, into virtual environments for video games and industrial digital twins.”
This new AI tool has not yet been combined with 3D printing applications, but an Nvidia spokesperson told the 3D printing industry that this could be a future direction. Currently, Neuralangelo models cannot be 3D printed.
The potential for additive manufacturing integration with Neuralangelo is easy to imagine.
Whether you want to export your AI-generated mesh as a future printable file or as part of a digital twin. Digital twins are now used in the design and prototyping stages of product development, and even into the product lifecycle.
Transform 2D videos into 3D models using AI
Neuralangelo employs Instant Neural Graphics Primitives (NGP) combined with Signal Distance Function (SDF) to achieve high quality video conversion. NGP provides a hybrid 3D grid structure with multi-resolution hash encoding. This hybrid representation greatly increases the expressive power of neural fields, allowing them to capture finer-grained details.
In effect, Neuralangelo takes a 2D video of an object or environment and selects several frames highlighting different perspectives. Once the camera position for each frame is determined, the AI creates a rough 3D model. This rough rendering is optimized to sharpen details and produce a finished product with high surface quality and accuracy.
The researchers highlight two key elements of this approach. First, compute higher derivatives using numerical gradients. This is said to be important for stabilizing the optimization process. A step-by-step optimization schedule is then essential to recover structures at different levels of detail. This enables comprehensive and accurate 3D model reconstruction.
Nvidia’s Neuralangelo project will be unveiled at this year’s Computer Vision and Pattern Recognition (CVPR) conference in Vancouver, June 18-22. Neuralangelo will be featured alongside nearly 30 other projects covering a wide range of topics ranging from pose estimation, 3D reconstruction, and video generation.
AI and additive manufacturing
Artificial intelligence is one of the main buzzwords in 3D printing right now. This year’s 3D Printing Industry Executive Survey focuses on automated AI 3D model generation as part of the future of 3D printing.
Will that future come sooner than expected? Nvidia has announced an AI 3D model generation tool called Magic3D. Magic3D creates a 3D object within 40 minutes of your text prompt. This is twice as fast as DreamFusion’s. Materialize has also released a metal 3D printing process control and build process software development kit. Materialize Process Control’s AI automates layer data analysis, saving you time and money.
In a 3D printing industry interview with Thangs founder Paul Powers, he discussed the future role of AI and 3D design.
subscribe 3D printing industry newsletter Make sure you get the latest 3D printing news.You can also follow twitterlike us Facebook Visit the page and subscribe. YouTube of the 3D printing industry Channels give you access to more exclusive content.
Interested in working in the additive manufacturing industry? Visit 3D printing jobs View a selection of available roles and kick-start your career.
The featured image shows a 3D model of Michelangelo using Neuralangelo. Image via NVIDIA.
