We see digital avatars everywhere from our favorite chat applications to virtual marketing assistants on our favorite e-commerce websites. These are becoming more and more popular and are quickly becoming part of our daily lives. Go to the avatar editor, choose your skin tone, eye shape, accessories, and more, and you’re ready to mimic you in the digital world.
Manually constructing a digital avatar face and using it as a living emoji can be fun, but it only scratches the surface of what is possible. The true potential of digital avatars lies in their ability to clone our entire bodies. This type of avatar is an increasingly popular technology in video games and virtual reality (VR) applications.
Generating high-fidelity 3D avatars requires expensive and specialized equipment. Therefore, it is only used in a limited number of applications, such as professional actors seen in video games.
What if we could simplify this process? Imagine being able to generate high-fidelity 3D full-body avatars using just a few real-world videos. No need for specialized equipment or complex sensor setups to capture every detail. Simply record with your camera and smartphone. This breakthrough in avatar technology could revolutionize many applications such as VR, robotics, video games, movies and sports.
The time has come. We have tools that can generate high-fidelity 3D avatars from videos shot in nature.time to meet vid 2 avatar.
Vid2Avatar learns 3D human avatars from real videos. No ground truth monitoring, priors drawn from large datasets, or external segmentation modules are required. Just give someone a video and it will generate a robust 3D avatar.
Vid2Avatar has some clever tricks to make this happen. The first thing we do is separate the human from the background in the scene and model it as a neural field. They solve the task of scene separation and surface reconstruction directly in 3D. They model his two separate neural fields, implicitly learning both the human body and the background. This is usually a difficult task as the human body needs to be related to his 3D points without resorting to 2D segmentation.
The human body is modeled using a single temporally consistent representation of human shapes and textures in standard space. This representation is learned from deformation observations using inverse mapping of parametric body models. Additionally, Vid2Avatar adjusts multiple parameters related to the background, the person and their pose using an optimization algorithm to best fit the available data from a series of images or video frames.
To further improve the separation, Vid2Avatar uses a special technique to represent the scene in 3D. This separates the human body from the background, making it easier to analyze each movement and appearance separately. We also use new objectives, such as focusing on a clear boundary between the human body and the background, to guide the optimization process to produce a more accurate and detailed reconstruction of the scene.
Overall, a holistic optimization approach for robust and high-fidelity human body reconstruction is proposed. This method uses the actual captured video without requiring any additional information. Carefully designed components result in robust modeling and the end result is a 3D avatar that can be used in many applications.
Please check paper and plan. All credit for this research goes to the researchers of this project.Also, don’t forget to participate 15,000+ ML SubReddits, Discord channeland email newsletterShare the latest AI research news, cool AI projects, and more.
Ekrem Cetinkaya graduated with a Bachelor of Science degree. He completed his master’s degree in 2018. He graduated in 2019 from Ožegin University, Turkiye, Istanbul. he wrote his master’s degree. A paper on image denoising using deep convolutional networks. He got his Ph.D. He completed his doctoral dissertation in 2023 at the University of Klagenfurt, Austria, titled “HTTP Adaptive Using Machine Learning to Enhance Video Coding for His Streaming”. His research interests include deep learning, his vision of computers, video encoding, and multimedia networking.
