A new study shows how unlocking a regular phone could eventually support passive heart rate tracking, using facial video and deep learning to make cardiovascular monitoring more accessible, while raising important questions about accuracy, privacy, and real-world clinical use.
Research: Passive heart rate monitoring during smartphone use in daily life. Image credit: Have a nice day Photo/Shutterstock
Smartphones will soon automatically track heart health while people use their smartphones normally, without the need for smartwatches, fitness trackers, or intentional heart rate checks. In a large study published in the journal natureresearchers created a passive heart rate monitoring (PHRM) system that uses a cell phone’s front camera to detect changes in blood flow in a person’s face.
The system then uses deep learning algorithms to analyze these changes and estimate the individual’s heart rate (Human resources department) and resting heart rate (RHR). Validated in over 160,000 videos, the system performed well in laboratory settings and everyday situations across skin tones, addressing the limitations of previous systems. Since smartphones are already widely used, this technology could make heart health tracking more accessible to many people, especially those who cannot afford smart wearable devices.
RHR This refers to the number of times the heart beats per minute when the body is at rest. This is an important measure of cardiovascular health. change in RHR It may indicate an increased long-term risk of heart disease. Measuring today RHR Usually, over time you will need a fitness band or smartwatch. However, not everyone uses these devices regularly. On the other hand, smartphones are widely used all over the world. What if you could measure it with your mobile phone? RHRcardiac tracking could become easier, more convenient, and more affordable for people from different socio-economic groups.
Scientists have so far performed remote photoplethysmography (rPPG) Measuring technology RHR using your smartphone’s camera. However, these studies included only small numbers of participants and tested the technology in controlled conditions. The system was also less accurate for people with darker skin tones. As a result, it remains unclear whether this technique can provide reliable measurements. RHR during everyday smartphone usage across diverse races and ethnicities around the world.
About research
In this study, the researchers PHRM The technology uses data from several laboratory studies and real-world studies conducted between 2020 and 2024. The system records a short 8-second video of a person’s face when they unlock their phone. A.I. An algorithm then analyzes these videos to determine the person’s Human resources department and RHR.
The team first trained the system to recognize it using 192,353 video recordings from 485 people. Human resources department pattern. They then tested the system using another set of 162,546 recordings collected from 211 different individuals of different ages, genders, body sizes, and skin colors. This diverse participant pool helped researchers assess whether the technology would work across a wide range of users. The researchers intentionally included more people with dark skin to address previous limitations in measurement accuracy.
The researchers used spectrocolorimeter measurements and the Fitzpatrick skin type classification to assess skin tone in the laboratory study, whereas participants in the free-living study had Monk’s skin tone (MST)scale.
of A.I. The algorithm verified the quality of the video input. Confidence-based gates were then used to reject low-quality measurements, such as recordings affected by poor signal quality, inadequate lighting, or excessive motion. The system then combined multiple measurements taken throughout the day to calculate a person’s weight. RHR.
Finally, the team compared smartphone measurements. electro-cardiogram recording of Human resources department And from wearables RHR Estimation to assess the accuracy of new ones PHRM system. They also investigated whether it was phone-based. RHR Correlates with established indicators of cardiovascular health.
result
of PHRM Accurately measured system Human resources department Both in controlled laboratory tests and in everyday normal use. It also produced the following accurate estimates: RHR. When researchers compared smartphone measurements to measurements obtained from a reference electro-cardiogram Similar results were observed when recordings were made. If valid Human resources department The measurements had an average absolute percentage error below the industry threshold of 10 percent across all three skin tone groups.
of PHRM The system met industry accuracy standards for consumer heart rate monitors. Comparison with wearables Human resources department tracker, PHRM Achieved daily average absolute error (before) is below the pre-specified goal of 5 beats per minute (BPM) whole. This system also beats 15 models with the following features: rPPG technology.
However, the system did not produce usable measurements from all videos, and the rate of valid video level measurements was lower for the darkest skin color group in free-living conditions.
for RHRparticipant-level accuracy for the darkest-skinned group initially did not meet the 5 bpm goal, but performance improved from day 3 onwards as the system’s filtering algorithm converged.
An important finding was that a high proportion of cases originated from smartphones. RHR It was associated with known cardiovascular risk markers, including higher body mass index and poorer cardiorespiratory fitness. Therefore, rather than demonstrating that the system can diagnose heart disease, the smartphone measurements confirmed the physiological validity of the approach.
Smartphone-based measurements were also more consistent from day to day than traditional one-time measurements RHR check. If this approach is further validated, it may be possible to monitor changes in cardiovascular health during daily smartphone use.
conclusion
The findings suggest that individuals could eventually be able to track their heart health simply by using their smartphones throughout the day. When people unlock their phones at different times, the phone can take multiple measurements and assess changes in heart rate patterns without having to wear a specific tracker or fitness device. Further work could optimize performance by reducing battery usage and testing on people with different heart conditions.
The researchers also emphasized that because the system involves passive facial video capture, actual use would require explicit informed consent, strong privacy protections, and secure on-device processing.
By making it publicly available, A.I. The researchers hope to use their model and research dataset to advance privacy-friendly heart rate monitoring technology that can expand access to cardiovascular monitoring through the smartphones people already use every day.
The study was conducted by researchers at Google Research and the University of Washington with funding from Alphabet or an Alphabet subsidiary, and the authors reported on Alphabet’s employment, potential stock ownership, and related patent filings.
Click here to download your PDF copy.
