By Anne H. Carlson
Although many sleep problems are multifactorial, artificial intelligence (AI) and machine learning models in sleep medicine have been primarily limited to inputs from a single data source. However, a trend in multimodal AI is AI trained to identify patterns across different data inputs, for example polysomnography (PSG) results. and New technologies such as socio-economic data are beginning to bring sleep into a real-world context.
“By addressing complex issues that can't be easily detected using single-modal information, such as pulse rate or oxygen levels, multimodal AI can provide more accurate diagnoses and treatment plans,” said Dr. Mikael Kågebäck, chief technology officer at sleep wellness app and smart alarm clock Sleep Cycle.
These findings may also lead to the development of sleep testing, tracking, and screening tools that incorporate multiple AI modalities for more targeted treatment.
Recent multimodal AI research has produced some surprising discoveries that will impact the future of sleep medicine. Here are a few highlights.
Predicting the spread of disease
Sleep-tracking app SleepCycle is used by millions of people in more than 150 countries and has collected more than 2 billion nights of data from opt-in customers since its launch in 2009. “By having access to this vast amount of sleep data from all over the world, we're able to answer questions that were previously unanswerable,” Kogeback says.
For example, in the summer of 2023, Sleep Cycle conducted a study to geographically identify the number of coughs among reported COVID-19 cases in the United States. The company's Cough Radar tool uses speech analytics and machine learning to track how many times a user coughs per hour and changes in coughing patterns. Researchers can see increases in coughing patterns even before people seek testing or treatment.
“Based on the increase in coughing, we were able to predict waves of the disease two weeks before cases were officially reported,” Kogebach says. The study also uncovered regional and clear seasonal patterns of the disease within the metropolis.
“Tracking how the disease spreads through cough data will help us understand it better and be able to better combat disease outbreaks in the future,” Kogeback says.
For its multimodal AI research, Sleep Cycle leverages its in-house collected data and extensive sleep medicine research environment.
“To improve our sleep tracking capabilities, we undertook the significant project of developing powerful respiratory tracking,” says Kogeback. “We integrated this data with motion data and trained a sleep stage model to be able to predict sleep recordings produced by PSG, the gold standard in sleep research. The result is a significant improvement in the accuracy of our audio-based sleep stages.”
Through its customer base, Sleep Cycle receives short audio clips of events such as breathing, snoring, movement and talking.
“Our team of annotators meticulously label these events, contributing to the creation of a comprehensive dataset that includes more than 300 different classes,” Kågebäck said.
Due to the complexity of conducting studies with large numbers of subjects, Kågebäck places emphasis on a research-based approach.
“When dealing with complex data sets that include many factors, it's important not to jump to conclusions based solely on apparent correlations,” says Kogeback. “It's important to make sure correlations are supported by solid research and evidence before proceeding.”
Sleep Cycle is currently exploring partnership opportunities with universities and other stakeholders to evaluate how cough data can contribute to early warning systems for disease outbreaks.
Practical testing for pediatric sleep apnea
Collecting children's sleep data at home poses several challenges, which is why at-home sleep apnea testing is far less common among pediatric patients than adults. In a recent study, Dr. Rahmatollah Beheshti, an assistant professor at the University of Delaware, and his team used machine learning to investigate how to more easily detect sleep apnea in children at home.
“One of the major obstacles to performing pediatric sleep apnea testing outside of a clinic is the presence of noise and gaps; children move a lot, are not very cooperative, pull on the probe, etc.,” Beheshti says.
Collecting sleep data at scale, especially for pediatric patients, is often the first hurdle: “AI models require large amounts of data, and access to sleep data (even if not consumer- or lab-level data) is difficult because of a variety of challenges, including privacy issues,” Beheshti notes.
In the study, the team used information from PSG data collected in a controlled laboratory study. They used machine learning-based models to analyze electrocardiograms (ECG), electroencephalograms (EEG), electrooculograms, and oxygen saturation (SpO2), carbon dioxide, and respiratory signals.
The machine split the signal into epochs collected from different sources (e.g., PSGs and data obtained from patients' electronic health records) and learned from them. “We also injected different kinds of manual noise into the data to bring the study closer to out-of-clinic scenarios,” Beheshti says.
The study showed that it may be possible to detect apnea in children with high accuracy using advanced AI algorithms, even without an EEG. “This was somewhat surprising to our team, since EEG is generally considered the most clinically significant signal in children,” Beheshti says.
“Two signals that are easy to collect at home (ECG and SpO2) can also achieve highly competitive results compared to using all six or seven modalities, which may address concerns about collecting different sleep signals from children outside the clinic.”
It is hoped that these findings will help improve access to pediatric sleep testing: “The most obvious use is to empower patients and families to consult with clinical experts at the appropriate time, so they can make more informed decisions,” Beheshti said.
Three recently published multimodal health data studies from EnsoData, developer of the AI-based EnsoSleep sleep scoring platform, focus on socio-economic factors that can impact obstructive sleep apnea (OSA) diagnosis and treatment, finding, for example, that in the most deprived areas, fewer than 50% of patients initiate treatment after being diagnosed with OSA.
“Identifying and better understanding gaps in access to OSA care can create opportunities to address and close those gaps,” said Chris Fernandez, chief research officer at EnsoData. “This requires clinical tools, testing, and treatments that are comprehensive and accessible to patients regardless of race, gender, age, income level, or location.”
Based on the records of more than six million patients, these studies linked claims data, diagnosis records, and medication information with socioeconomic measures such as the Area Poverty Index mapping tool, which shows the relative socioeconomic status of a given area. The studies also considered demographic factors such as race, gender, and age, and examined disparities in treatment and income.
“We believe that social determinants of health variables should be part of the validation of any AI/machine learning models proposed for population health applications,” Fernandez says.
Fernandez noted that AI is already helping sleep medicine move toward precision medicine by streamlining patient testing, helping doctors scale the diagnostic process, and identifying patients who struggle with CPAP.
“Leveraging multimodal biomedical AI models to analyze sleep studies has the potential to transform our understanding of sleep health and disorders and their broader role and impact on health risks, longevity and quality of life,” Fernandez says.
figure 141540838 © Joseph Bagota | Dreams Time
