This prospective, single-centre, shadow-mode diagnostic accuracy validation study evaluated a publicly available DenseNet-121 convolutional neural network (TorchXRayVision) for triage of chest radiographs in a health-assessment population. The investigation was reported according to STARD 2015 and STARD-AI/DECIDE-AI guidance. The prespecified aim was to assess diagnostic accuracy metrics with an emphasis on achieving a sensitivity criterion (≥95%) for abnormality detection.
The study took place in the Department of Radiology and Imaging at Patan Academy of Health Sciences/Patan Hospital, Lalitpur, Nepal. The cohort comprised 826 consecutive health-assessment applicants undergoing chest radiography for foreign employment predeparture examinations and student migration screening from 5 June 2026 to 20 June 2026 inclusive (16 days). Two radiographs were excluded because of Digital Imaging and Communications in Medicine (DICOM) technical failure, leaving 826 evaluable cases per the report.
The index test was a publicly available DenseNet-121 model from the TorchXRayVision library using densenet121-res224-all pretrained weights. For each radiograph, a single aggregated pathology probability score was derived. A post hoc threshold of 0.6258 was selected as the highest threshold meeting the prespecified ≥95% sensitivity criterion. The threshold selection and all reported model outputs were derived and assessed within the shadow-mode validation framework described.
The reference standard was a single-reader-per-case review performed by one of three radiologists blinded to the AI output: two board-certified radiodiagnosticians (LS reviewed 276 cases; DM reviewed 275 cases) and one radiology resident (UB reviewed 275 cases). Each reader used a standardised data collection worksheet to record a binary classification (abnormal or normal) and free-text findings. According to this reference standard, 41 of 826 radiographs (4.97%) were classified as abnormal.
At the post hoc threshold of 0.6258 the DenseNet-121 algorithm produced the following primary results against the single-reader reference standard:
The authors note that the high NPV supports potential use of the algorithm as a radiographic abnormality rule-out triage tool in this low-prevalence screening population; however, they explicitly state this does not equate to microbiological exclusion of active pulmonary tuberculosis.
Despite preserved discrimination (high AUC), the algorithm exhibited calibration failure. Reported calibration metrics included a Brier score of 0.3621 (null Brier 0.0472) and an expected calibration error of 0.564. The study observed systematic score compression, with model scores concentrated in a narrow range (0.52–0.72). These findings indicate a distributional shift affecting calibration when the pretrained model was applied to this local population.
Ten-fold cross-validation was performed to assess internal robustness. Cross-validated, bias-corrected results reproduced the primary sensitivity estimate (95.12%; 95% Wilson CI 83.9% to 98.7%; optimism 0.00 percentage points) and yielded specificity 75.80% (95% Wilson CI 72.7% to 78.7%; optimism +1.40 percentage points). These internal validation results suggest that the reported primary metrics were not materially inflated by circular optimisation within the study dataset. The study did not include independent external validation.
The report highlights an important limitation: only 41 reference-standard positive cases were present, producing a wide Wilson confidence interval for sensitivity (width 14.8 percentage points, 83.9%–98.7%) and substantial uncertainty around the point estimate. The authors caution interpreting the sensitivity estimate in light of this small number of positives.
The DenseNet-121 algorithm showed a high point-estimate sensitivity and excellent discrimination for chest radiograph triage in this Nepali health-assessment cohort, and its very high NPV suggests potential utility as a rule-out triage tool. However, systematic calibration failure manifested as score compression despite preserved discrimination, which the authors describe as a quantifiable marker of low- and middle-income country distributional shift when applying pretrained models.
Before any operational deployment, the authors recommend prospective local calibration studies and independent external validation to address calibration issues and confirm performance across different populations and clinical workflows. They also reiterate that a radiographic triage pass does not replace microbiological testing when infectious disease exclusion is required.