COMMERCIAL AI tools designed to detect lung cancer features on chest radiographs showed striking differences in diagnostic performance in a UK head-to-head comparison, raising questions about how hospitals should select systems for clinical deployment.
Lung cancer remains one of the leading causes of cancer death globally. Chest radiography is often the first imaging investigation used when symptoms are suspected, making accurate interpretation critical for early diagnosis and referral.
The study evaluated seven commercially available AI devices using more than 5,235 chest radiographs from primary care patients at a single UK centre. Researchers found substantial variation in sensitivity, specificity and false-positive rates between products, despite all being designed to identify lung cancer on routine imaging.
AI Deployment Arrives Amid Radiologist Shortages
The findings come as health systems turn to AI-assisted imaging to manage workforce pressures. In the UK, the National Optimal Lung Cancer Pathway recommends chest radiographs from primary care are reported within 24 hours. However, a 29% shortfall in radiologist numbers has contributed to reporting delays.
AI tools have been proposed as a way to prioritise urgent scans, support same-day CT referral decisions and improve detection of subtle early-stage tumours that may be missed on radiographs.
In 2023, the UK government allocated £21 million to 12 imaging networks in England to support AI deployment, including tools for lung cancer detection. However, the National Institute for Health and Care Excellence previously concluded there was insufficient evidence to support routine clinical adoption.
Large Differences in Lung Cancer Detection Rates
The retrospective study included 5,235 consecutive adult patients who underwent chest radiography between July 2020 and February 2021. Median patient age was 60 years, 53.4% were women and 79.4% were White. Lung cancer with a visible tumour was confirmed in 1.4% of patients using multidisciplinary team diagnosis as the reference standard.
Each radiograph was independently analysed by all seven AI systems. Diagnostic accuracy differed considerably across devices. Area under the receiver operating characteristic curve ranged from 0.80 to 0.94. Sensitivity varied from 20.8% to 77.8%, while specificity ranged from 58.9% to 98.4%. Positive predictive value ranged between 1.5% and 28.4%.
Compared with radiologist reports, three devices identified more tumours, while four detected fewer. Additional false-positive findings ranged from 10 to 2,039 cases depending on the system evaluated. Agreement between devices was limited, with minimal concordance in classification results.
Findings May Influence NHS AI Adoption
The authors said the variability between systems could affect how hospitals and imaging networks evaluate AI tools for chest radiography. Although some devices detected more tumours than radiologists did, this often came with substantially higher false-positive rates that could increase unnecessary follow-up imaging and workload.
The findings also suggest that AI systems designed for the same clinical task should not be considered interchangeable. Minimal agreement between devices indicated that different platforms frequently flagged different patients as suspicious for lung cancer.
The study comes as the NHS expands investment in imaging AI, despite limited evidence supporting routine use in clinical practice. The researchers said further studies are needed to assess how these tools influence radiologist performance, patient outcomes and service delivery before wider adoption.
Reference
Maiter A et al. Independent head-to-head comparison of commercial artificial intelligence devices for lung cancer detection on chest radiographs. Radiology. 2026;DOI:10.1148/radiol.252205.
Featured image: vanzittoo on Adobe Stock
