According to the NVIDIA technical blog, Reallusion is integrating NVIDIA's advanced AI technology to revolutionize digital character creation. The collaboration is set to transform animation workflows for filmmakers, game developers and content creators.
AI-Driven Animation with Audio2Face
Reallusion leverages NVIDIA's Audio2Face technology, which automatically generates expressive facial animation and lip sync from audio or text input. Supporting multiple languages, Audio2Face can be used to animate characters as they speak or sing, making it a versatile tool for animators. The latest standalone release also includes the ability to animate realistic facial expressions, with detailed adjustments using sliders and keyframe controls.
Integrated into Reallusion's Character Creator and iClone applications, Audio2Face enables a seamless AI-assisted animation workflow, allowing users to prepare assets for animation with a single click and generate live facial movements matching the provided audio tracks. The generated animations can then be refined in iClone before rendering for use in a wide range of production environments.
Streamlined Animation Workflow
NVIDIA and Reallusion collaborated to develop the CC Character Auto Setup plug-in, which streamlines a previously cumbersome 18-step process into a single operation. Users can import Character Creator assets, select a training model, and bring their 3D characters to life with realistic facial animations synced to audio input. Audio2Face's motion sliders and keyframe controls allow for additional performance sculpting before making final production adjustments in iClone.
iClone provides precise control over every aspect of facial animation, from expression levels to head movement and simulated eye movements, allowing animators to faithfully express a character's personality. The software also allows you to incorporate head movements captured from motion capture equipment such as AccuFACE or iPhone Live Face.
AccuFACE: Next-Gen AI Facial Motion Capture
Powered by NVIDIA Maxine AR SDK, AccuFACE delivers real-time facial capture quality and capabilities. Powered by NVIDIA GPUs with Tensor Cores, Maxine AR SDK provides AI-driven 3D face tracking, body pose estimation, and more. AccuFACE transforms captured facial data into seamless digital animation, enabling the generation of expressive facial animations and responsive 3D avatars in real time.
Key features of AccuFACE include precise landmark mapping, head pose and deformation tracking, facial mesh reconstruction, and robust face detection and localization, allowing it to capture subtle facial expressions that are essential for conveying emotions and enhancing the believability of digital characters.
AccuFACE also provides tools to refine the AI-generated tracking for professional results. Device settings like smooth filtering and noise removal combat tracking artifacts, while anti-interference cancellation prevents false cross-triggering of facial movements. Further tuning and refinements allow you to capture clear facial expressions and achieve lifelike performances.
availability
NVIDIA Maxine delivers high-quality video communications and AI technology for professionals. The latest Maxine product release is included in NVIDIA AI Enterprise, providing access to production-ready features and enterprise support. For early access to new features, users can join the Maxine Early Access program.
“Reallusion's partnership with NVIDIA demonstrates the transformative potential of AI in animation, making professional facial motion capture and animation accessible to a wider audience. This advancement will revolutionize digital character animation by enabling animators to achieve high-quality results without the need for advanced expertise or specialized equipment.”
For more information, see the NVIDIA technical blog.
Image credit: Shutterstock
. . .
tag
