Augmented reality (AR) is a digital media platform that allows users to integrate virtual contexts into physical environments in an interactive and multidimensional way.
Implementing AI enables deep neural networks to replace traditional computer vision approaches and add new capabilities such as object detection, text analysis, and scene labeling to enhance AR experiences. Explore AI in AR, its applications, examples, and vendors.
How will AI transform AR?
Historically, AR software used a traditional computer vision technique called Simultaneous Localization and Mapping (SLAM). SLAM algorithms compare visual features between camera frames to map and track the environment.
However, modern AR applications rely on deep learning to provide more advanced features. AR developers can leverage AI algorithms to deliver AR features such as enhanced interaction with the surrounding physical environment. AI technologies such as machine learning, GenAI, and deep learning are well suited for AR environments for the following reasons:
- Since the camera is always on, there is an opportunity to collect more data for training the AI algorithm.
- Because AR environments rely on multiple sensors (device gyroscopes, sensors, accelerometers, GPS, etc.), the inputs to AI algorithms are rich in detail. This makes it more reliable than systems that rely on only a single sensor.
In parallel with deep learning, AR systems are increasingly using spatial intelligence to combine semantic segmentation, depth estimation, and context modeling to understand the entire environment, not just objects. This allows AR content to behave physically realistically (occlusion, anchor shadows, lighting adaptation, etc.) and enables advanced features such as contextual recommendations based on scene category (office vs. outdoors) and inferred user intent.
8 AI applications in AR
1. Labeling objects
Machine learning classification models are used to label objects. When a camera frame is executed in the model, the image is matched against predefined labels in the user’s classification library and the labels are overlaid onto physical objects in the AR environment. For example, Volkswagen Mobile Augmented Reality Technical Assistance (MARTA) labels vehicle parts and provides information about existing problems and how to fix them.
2. Object detection and recognition
Object detection and recognition utilizes convolutional neural network (CNN) algorithms to estimate the location and extent of objects in a scene. Once an object is detected, the AR software renders the digital object to overlay the physical object and mediates the interaction between the two. For example, the IKEA Place ARKit application scans the surrounding environment, measures vertical and horizontal surfaces, estimates depth, and suggests products that fit a specific space.
If you want to learn more, check out our article on image recognition.
3. Text recognition and translation
Text recognition and translation combines AI optical character recognition (OCR) technology with text-to-text translation engines such as DeepL. A visual tracker tracks words and allows translations to be overlaid onto the AR environment. Google Translate provides this functionality.
4. Automatic speech recognition
Automatic speech recognition (ASR) uses neural network audiovisual speech recognition, an algorithm that relies on image processing to extract text. Typing a specific word triggers an image in the library labeled to match that word’s description, and projects that image into AR space. An example is the Panda Stickers app.
To learn more, read our collection of top use cases for speech recognition.
5. Gestures and natural interactions
AI-powered gesture tracking and multimodal interaction allow AR systems to recognize hand, body, and finger movements in real time. Combining these systems with voice AI allows users to interact with virtual objects without touching them, creating a more intuitive and hands-free AR experience.
example:
In industrial maintenance, AI AR systems can interpret hand signals to interact with 3D holograms on machines, and voice commands can trigger context-sensitive instructions and alerts. Accessibility-focused AR apps use gestures and voice to navigate interfaces for users with limited mobility.
Usage example:
- Industrial AR applications for hands-free equipment control
- Accessibility apps that provide gesture-based navigation and commands
- Games and entertainment that control virtual objects with gestures
- AR training and simulation environment with natural interactions
6. Environment mapping and scene understanding
AI goes beyond simple object detection and enables semantic scene understanding, allowing AR systems to classify entire environments (kitchens, offices, streets, etc.) and adapt overlays accordingly. Deep learning models like SceneNet and IBM’s Visual Recognition can analyze spatial context, lighting, and surface type to tailor AR experiences.
example:
Snapdragon Spaces uses AI to detect walls, surfaces, and room types in real-time, allowing you to place virtual furniture and game elements more realistically.
Usage example:
- Interior design app that recommends furniture based on room type
- AR wayfinding that adapts signs to indoor/outdoor environments
- Smart retail that changes promotional content depending on store section.
7. Generative AI for dynamic content creation in AR
GenAI models can dynamically generate 3D assets, audio, and even entire scenes based on prompts and user interactions within the AR environment. This removes the need for preloaded libraries and opens the door to personalized, real-time worldbuilding.
example:
A marketing app will allow users to describe their ideal living room, and GenAI will generate furniture and layouts in AR.
Related models/tools:
- Luma AI (Text to 3D)
- RunwayML for video overlays
- Pika Labs or Spline for real-time 3D modeling
8. Anomaly detection for industrial inspection
AI-enabled AR helps in real-time anomaly detection in manufacturing and in the field. A computer vision model trained on what is “normal” (such as the integrity of a pipe or the surface of a machine) can detect deviations and highlight them in the user’s view using AR.
example:
Porsche uses AI inspection tools and AR to highlight wear, corrosion, and misalignment of car parts during remote maintenance.
Usage example:
- Maintenance and safety inspection within the factory
- Public infrastructure (power lines, pipelines, etc.)
- Aircraft or vehicle repair evaluation
Increase in AI/AR applications in various industries
AR is used in many applications, especially in entertainment and construction. Other industries that can benefit from AI/AR include:
- construction: Architecture, design, project planning, site modification, safety and inspection, underground construction, training.
- education: Field exploration (museums, factories), model experiments in laboratories (chemistry, physics, geometry, anatomy)
- Entertainment: Real-time information from sports arenas, expanded music concerts, interactive ads, movies, and games.
- medicine: These include diagnosis, surgical navigation, training surgeons on new procedures, and modeling the effects of drugs.
- logistics: Warehouse planning and operation, transportation optimization, inventory management
- Manufacturing: Design and prototyping, maintenance, repair, training,
- Army: Aircraft navigation, weapon targeting, and telepresence in military operations.
- real estate: Marketing, interior design, floor planning, and construction staff training.
- fashion: Try before you buy, in-store navigation, personalized shopping, AR window shopping, and makeup apps.
AI-enabled AR software vendors
The global market for augmented reality (AR), virtual reality (VR), and mixed reality (MR) is estimated to reach $100 billion by 2026, according to Statista. Companies like Apple and Google are entering the market to develop AI-enabled AR software to improve the AR experience for their customers.
Here are the top AI-enabled AR software vendors:
Apple ARKit
ARKit is Apple’s augmented reality (AR) development platform for iOS iPhone and iPad. ARKit provides object labeling, person occlusion, motion capture, and multiple face tracking. ARKit is used in:
- education To model hands-on experiments in science, physics, or chemistry laboratories such as Labster
- construction and architecture To measure space dimensions and suggest products and solutions, such as IKEA Place.
- entertainmentPokemon GO, etc.
Google AR Core
ARCore is Google’s AR platform, which integrates digital content into physical environments through motion capture and object detection and recognition. ARCore is used in:
- real estate Visualize, decorate and design your free space. Sotheby’s Curate App and more
- Lifestyle and maintenance Connect users with experts who provide guidance and direction, take measurements, and project potential solutions into the space. For example, the Stream app.
- entertainment TendAR virtual pet game app and more.
others
Other AI/AR software vendors include:
- amazon sumerian
- microsoft mesh
- unity
- vforia engine
- zapworks
AI AR wearable and XR platform
Beyond SDKs, hardware platforms are now integrating deep AI directly into AR wearables. For example, devices like Apple Vision Pro provide spatial computing with hand, eye, and voice input to power contextual intelligence and AR interactions.
Meta’s Ray‑Ban Display smart glasses and other lightweight AI‑AR wearables bring contextual overlays, live translation, and interactive visual guidance to everyday use cases. These wearable platforms represent a transition from phone-centric AR to immersive, always-on AI AR experiences.
principal analyst
Sem Dilmegani
principal analyst
Cem’s work has been cited by major global publications such as Business Insider, Forbes, and the Washington Post, global companies such as Deloitte and HPE, NGOs such as the World Economic Forum, and supranational organizations such as the European Commission. See more reputable companies and resources that reference AIMultiple.
Throughout his career, Cem has worked as a technology consultant, technology buyer, and technology entrepreneur. He has spent more than a decade advising companies on technology decisions at McKinsey & Company and Altman Solon. He also presented a McKinsey report on digitalization.
Reporting to the CEO, he led the communications company’s technology strategy and procurement. He also led the commercial growth of deep tech company Hypatos, from 7-figure annual recurring revenue and 0 to 9-digit valuation within two years. Cem’s work at Hypatos has been featured in major technology publications such as TechCrunch and Business Insider.
Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.
View full profile
