Apple’s Vision Pro uses machine learning to create FaceTime avatars

• Apple’s Vision Pro allows users to create digital avatars for more realistic video chats.
• The tech giant said it uses “advanced machine learning,” a form of AI, to represent movement.
• An Apple executive said video conferencing was one of the “most difficult challenges” the team faced.

Apple’s Vision Pro headset allows users to create digital avatars, allowing for more realistic video chats even when part of the face is covered.

The tech giant said it uses “advanced machine learning” to realistically represent a user’s face and hand movements during FaceTime chats. Users need to scan their face with the headset’s front-facing camera to create a highly realistic 3D digital avatar.

Apple demoed the feature in an introductory video, saying the system would allow users to see “eyes, hands, and real facial expressions” during video chats.

Apple unveiled its first flagship product in eight years, the headset, at the Apple Worldwide Developers Conference on Monday. The headset was highly anticipated and received mostly positive reviews from critics despite its high price tag.

During the conference, Mike Rockwell, head of Apple’s AR/VR project team, said that the “most difficult task the team faced in building Vision Pro was because users always have something over their eyes.” One of the “challenges,” he said, was video conferencing.

Apple representatives did not immediately respond to Insider’s request for more information about the feature.

From the use cases in the video, Apple’s augmented reality headset avatars look very realistic, which is very different from Meta’s early virtual reality avatar attempts. His CEO of Meta, Mark Zuckerberg, was widely mocked in his early days in the Metaverse for a scathing meme shared in response to his avatar selfie.

Last year, Zuckerberg shared a photo of himself in front of the Eiffel Tower on Facebook. It was quickly accepted by social media users for its basic graphics. The CEO has since posted a more evolved version of his avatar on Instagram as an obvious response to the criticism.