In a systematic review and meta-analysis published in JAMA DermatologyLaiouar-Pedari et al. evaluated the real-world diagnostic performance of artificial intelligence (AI)-assisted dermoscopy for melanoma detection. This study was conducted to address a significant gap in the literature. Previous retrospective studies have suggested that AI may match or exceed dermatologist performance, but prospective evidence more reflective of clinical practice has been limited. The authors focused only on prospective studies and sought to determine whether AI is suitable for routine clinical use and whether it can meaningfully improve clinician performance in melanoma diagnosis.
Research details
The analysis included 11 prospective studies consisting of more than 2,500 patients and more than 50 dermatologists. Eligible studies used dermoscopy imaging to evaluate adult patients with suspected melanoma, with histopathology as the reference standard. The researchers compared three diagnostic approaches: dermatologists alone, AI alone (mainly convolutional neural network-based systems), and dermatologists assisted by AI.
Data were systematically extracted and pooled for sensitivity, specificity, precision, and balanced precision. Studies were required to include at least 20 histopathologically confirmed melanoma cases and to use a prospective design excluding retrospective datasets and non-dermatoscopic imaging modalities. Risk of bias was assessed using the QUADAS-2 and QUADAS-C tools, revealing frequent concerns related to patient selection and study design, particularly the preselection of lesions suspected of melanoma and the use of a simplified binary classification system.
Main results
Across studies, dermatologists achieved an overall sensitivity of 78.6% (95% confidence interval). [CI] = 67.5% to 88.1%), specificity 75.2% (95% CI = 63.3% to 84.3%). The AI system showed comparable performance with sensitivity of 80.9% (95% CI = 63.6% to 94.5%) and specificity of 75.6% (95% CI = 64.5% to 85.6%). A single study evaluating AI-assisted dermatologists showed even better performance, with sensitivity of 91.9% and specificity of 83.7%. Direct comparisons suggested that AI offers higher specificity with similar sensitivity and may reduce unnecessary biopsies. However, variation between studies is high and many designs introduce biases, most notably due to limited patient populations and binary diagnostic frameworks that do not reflect real-world clinical complexity.
According to the study authors, the findings show that AI can achieve dermatologist-level diagnostic accuracy in prospective settings and may improve performance when integrated into clinical workflows, although current evidence remains preliminary.
“In systematic reviews and meta-analyses in predictive settings, AI systems perform at a comparable level to dermatologists for melanoma diagnosis and may improve performance when used as decision support tools. However, the frequent risk of bias and limited generalizability of current studies highlight the need for broader validation in unselected patient populations in clinical practice,” the researchers concluded.
Dr. Sarah Lyor Pedariis the corresponding author from the German Cancer Research Center, Heidelberg, Germany. JAMA Dermatology article.
disclosure: This study was funded by the Ministry of Health, Social Affairs and Integration of the State of Baden-Württemberg, Stuttgart, Germany. For full research author disclosures, please visit jamanetwork.com.
