Performance of the ‘DermDx’ algorithm in triaging suspected skin cancers: results of the AI-SCSS study

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Abstract Background The evidence base for the use of artificial intelligence (AI) in skin cancer diagnosis consists largely of artificial ‘reader’ studies and incomplete postimplementation survey data, leading to the National Institute for Clinical Excellence placing restrictions on use in the UK and raising questions regarding the value of AI as a medical device vs. traditional human teledermatology. Objectives To assess the real-world performance of a convolutional neural network-based system (‘DermDx’; MetaOptima Technology) in triaging skin lesions referred to secondary care on a UK suspected skin cancer pathway. Methods This was a blinded observational study to evaluate the performance of an AI algorithm on a consecutive series of referred patients with skin lesions of concern (March–April 2024). It was conducted at a single-centre secondary care National Health Service dermatology department in Cornwall, UK. All lesions referred from primary care on a suspected skin cancer pathway and imaged in a community lesion imaging clinic were eligible for inclusion. Exclusion criteria included the absence of research consent; acral, mucosal, umbilical and eyelid lesions; and inadequate image quality or artefacts. Dermoscopic images were captured with an Apple iPad and a DermLite DL4 dermatoscope, and dermoscopic images were assessed retrospectively in a blinded manner using the DermDx algorithm. Notes review was performed at 1 year to identify misdiagnoses. The main outcome measure for this study was to determine the sensitivity for detection of malignant and premalignant lesions. Results A total of 1024 lesions were included; 298 (29.1%) lesions were malignant, 271 (26.5%) were premalignant and 455 (44.4%) benign. The algorithm categorized 273 (26.7%) as low risk and 751 (73.3%) as high risk. Sensitivity for malignant and premalignant lesions was 98.1% [n = 558/569, 95% confidence interval (CI) 96.5–99.0]. Specificity for malignant and premalignant lesions was 57.6% (n = 262/455, 95% CI 53.0–62.0). The area under the receiver operating characteristic curve was 0.918 (95% CI 0.900–0.935). DermDx missed 2 of 298 (0.7%) skin cancers. Dermatologists missed 5 of 298 (1.7%) skin cancers (P = 0.25). Use of DermDx as a triage tool would have reduced face-to-face consultations by 26.7%. Conclusions The DermDx algorithm performed extremely well as a binary classification tool, with sensitivity for malignancy at least as good as that of dermatologists in this real-world setting.