AI transforms tourism for visually impaired users

Breakthrough AI changes how visually impaired people experience the world, providing them with tools to discover, understand and experience the beauty of unfamiliar places like never before.

Research: AI systems promote blindness and low vision when interpreting and experiencing unfamiliar environments. Image credit: Angel Santana Garcia/Shutterstock.com

A team of researchers in China has developed an artificial intelligence (AI) driven system that could help visually impaired individuals explore, understand and taste the unfamiliar environments around them. This study is published in Nature Portfolio JournalArtificial Intelligence.

background

Exploring natural environments such as parks has a major positive impact on your physical and mental health. However, people with poor vision and blindness are often excluded from these benefits.

Existing support solutions developed to guide visually impaired individuals primarily focus on providing functional assistance such as navigation and obstacle avoidance, allowing them to engage naturally and passively.

Visually impaired individuals often feel helpless as they explore unfamiliar environments. This usually means they rely on family, friends, or volunteers to seek support. This impairs the ability to actively explore and understand unfamiliar environments, meaning that you remember and communicate with other visually impaired individuals about their journey.

A team of researchers based in China developed an AI-driven system called Viptour, providing visually impaired individuals with a sense of independence in an unfamiliar environment.

How does Viptour work?

Viptour is an AI-driven system that includes a set of lightweight, portable, consumer-grade devices (camera and smartphone) and a new deep learning algorithm network called FocusFormer. Efficient multisensory interaction techniques, such as audio and hierarchical haptic interactions, promote interaction between visually impaired users and Viptour systems.

Focusformer views aesthetics, freshness (novelty), and basic needs (including navigation and safety) as the main factors that extract meaningful information from complex and unfamiliar environments and exclude redundant visual details. This reduces the cognitive load for visually impaired users.

Focusformer transforms huge amounts of information into structured, sparse, hierarchical, personalized graphs. Based on this well-structured graph, FocusFormer interacts with visually impaired users via smartphone application, understand their preferences, and provide personalized assistance via adapters.

It is trained with thousands of public tourism videos from sighted tourists, with self-monitoring methods beneficial to effectively reduce aesthetic bias.

The Viptour system also offers the option to record, store and share experiences, as well as to promote emotional communication between visually impaired individuals and the exchange of knowledge and experience within social networks.

Viptour's core innovation lies in the multi-attention Focusforfer network. This approach utilizes background subnetworks to exclude commonly seen objects, attraction subnetworks to identify highlights, freshness subnetworks to discover novel features, and subnetworks trained in surveys conducted with visually impaired participants. These subnetworks are combined to select, rank and present the most relevant information for each user.

The Viptour system also utilizes a BLV-in-the-Loop adapter, which allows for personalization by updating recommendations in real time based on individual user feedback such as “likes” and “dislikes.”

User opinions about Viptour

The Viptour system was tested in 33 people with blindness or low vision, and self-reported emotional experiences were collected for analysis.

With regard to support performance, this study found that the Viptour system effectively supports visually impaired individuals in actively exploring unfamiliar environments, fully understanding, reinforcing accurate and long-lasting recollections, and enabling peer communication.

By analyzing the self-reported experiences broadly, this study found that participants using Viptour successfully achieved a 67.9% increase in positive emotional responses, a 94.7% increase in arousal, a 772.73% increase in cognitive mapping accuracy, and a 200% increase in long-term memory accuracy.

User ratings showed that Viptour System usability scores exceed 80 out of 100, which are comparable or higher than other support tools for visually impaired individuals.

Physiological measurements, including electrode activity and heart rate fluctuations, show significant improvements with the use of Viptour, indicating enhanced emotional engagement.

The importance of research

This study emphasizes the potential uses of AI-driven Viptour systems to actively explore unfamiliar environments, providing visually impaired individuals with fun and memorable experiences. These experiences can greatly enhance your emotional state and improve your overall quality of life.

Existing evidence suggests that by presenting organized and engaging information, it increases the level of person's pleasure and promotes deeper memory retention. Humans have a natural tendency to process well-structured and meaningful information, which makes their experiences more enjoyable and memorable.

This human tendency may be explained by the concept of cognitive flow ency. This indicates that clear and organized information presentation reduces individual cognitive load. This will then help guide mental resources towards understanding and integration of content. This improved processing flow ency will trigger a positive response, as individuals perceive information more comfortably.

Furthermore, the interaction between novel and familiar information affects the effect of organized and interesting information on memory. New information stimulates curiosity and raises attention, while familiar information provides cognitive comfort and consistency.

Presenting information in a structured and engaging way can balance novelty and familiarity, helping to maintain personal interest and engagement.

Self-monitored training of focused forkers with thousands of unsigned public tourism videos effectively captures cognitive flow ency and reveals statistical relationships between different concepts of the tourism scene. This approach eliminates the potential bias of tour-first labeling, trains the model to extract only relevant context information.

These personalized design considerations from Focusformer allow the Viptour System to successfully model the desired cognitive flow ency, improving the tourism experience for the visually impaired.

It is worth noting that the impact of Viptour depends on the quality of the underlying AI technology, such as object detection and semantic graph generation. Future improvements in these methods could further improve system performance.

Download the PDF copy now!