Primary care physicians have significantly improved their accuracy in identifying underlying diseases.
These findings come from a recent web-based reader survey authored by researchers including Elizabeth V. Seiberling, MD, of the Department of Dermatology at the University of Pittsburgh, Pennsylvania. Seiverling and co-authors highlighted recent advances in AI technology in melanoma detection and the potential of such technology for primary care physicians.2
“Here we describe the results of a multireader multicase (MRMC) study to evaluate the referral performance of PCPs in evaluating lesions suggestive of melanoma with and without the use of the ESS device,” Seiverling et al. write.1 “Previous studies were not sufficiently powered or designed to demonstrate the impact of the device on melanoma management, so this study was accomplished by intentionally including more melanoma cases.”
Design and findings regarding the use of ESS devices
Researchers investigated an investigational handheld, non-invasive device that utilizes ESS combined with machine learning (ML) to assist clinicians in evaluating suspicious skin lesions. The development of the device’s algorithms implemented a dataset consisting of more than 10,000 ESS recordings collected from more than 2,000 skin lesions. These included both malignant and benign lesions.
Seiverling and his co-authors emphasized that the spectral data used to train the algorithm was completely separate from the dataset utilized in the current study. This ensured the researcher that training records were not included in the testing phase. The ESS device produces a binary classification result for each lesion analyzed. Lesions that exhibit malignant features are labeled “further investigation,” while lesions that exhibit benign features are given a “watch” notice.
When “further investigating” a lesion, the AI-powered device is designed to generate a spectral similarity score ranging from 1 to 10. In this range, the higher the score, the greater the similarity between the spectral profile of the lesion and the spectral profile of lesions known to be malignant from the algorithm’s development set.
The study by Seiverling et al. used a web-based multireader multicase (MRMC) reader study design for melanoma detection. Primary care physicians participated in a total of 200 readings per physician, which were performed on 100 unique lesion cases. Each such case was presented twice, the first time containing only standard clinical information and digital images, and the second time adding the output of the ESS device.
This analysis investigated two main aspects of clinician decision-making. The first is whether the lesion should be referred to a dermatologist for further evaluation, and the second is whether the lesion is determined to be malignant or benign. The analysis involved 118 board-certified internists and family medicine physicians who each evaluated 50 malignant and 50 benign lesions.
A study by Seiverling et al. found that a total of 5900 assessments were performed without ESS data and an additional 5900 with the assistance of an ESS device. In addition to diagnostic ability, the research team surveyed patients evaluated in this study about their confidence in clinical decision-making when implementing this AI-powered tool.
This analysis successfully met its primary endpoint, achieving an area under the receiver operating characteristic curve (AUROC) of 0.671 (95% confidence interval) for physicians using the device. [CI]: .611–.732), compared to an AUROC of .630 (95% CI: 0.582–0.678) without device assistance. Seiverling et al. note that this is a statistically significant improvement (P = .036).1
Overall, these findings demonstrated improved diagnostic accuracy for physicians as a result of access to ESS device data. When asked whether the device added value to clinical decision making, 91.5% of participating physicians agreed or strongly agreed, suggesting a high level of perceived clinical usefulness.1 In summary, the data suggest that the integration of ESS technology and machine learning could greatly assist primary care physicians in the early identification and management of lesions indicative of melanoma.
“Given the rising incidence of skin cancer and limited access to dermatological treatments in the United States, this device could have a significant impact on skin cancer detection in primary care settings,” Seiberling and colleagues concluded.1
References
-
Seiverling E, Shah A, Siegel D, et al. Improving diagnostic accuracy in primary care: A multireader multicase (MRMC) study of an AI-powered handheld elastic scattering spectrometer for informed referral decisions in melanoma evaluation. J Clin Aesthet Dermatol. 2025;18(10):59–65.
-
Garrow DS, Correa da Rosa J, Jaegerman S et al. Digital imaging biomarkers feed machine learning for melanoma screening. Expdermatol. 2017;26(7):615-8.
