The investigators sought to test the robustness of using late-stage cancer incidence as an earlier endpoint in cancer-screening trials by auditing the evidence base underlying a prior 41-comparison review. They reconstructed the PubMed corpus that supported the original review and then applied a blended approach in which an LLM (Gemini 3.1 Pro Preview) was used exclusively to prioritize candidate reports for human reassessment. All downstream determinations — eligibility, linking multiple reports from the same trial, harmonizing endpoint definitions, and verifying comparison-level data — were completed by human reviewers in a blinded process.
Gemini 3.1 Pro Preview was used only to sort or prioritize reports; it did not make eligibility or data decisions. Human reviewers accessed prioritized records, retrieved and assessed PDFs, reconciled documents that came from the same trial, and harmonized endpoints so that comparisons were compatible at the trial level. The workflow emphasized source verification and human adjudication rather than automated extraction or automated inclusion decisions.
Reviewers evaluated whether reports represented the same trial, whether trial comparisons were compatible with the endpoint definitions used in the source review, and whether the trial-level data reported were verifiable from source documents. When multiple reports derived from the same trial were present, reviewers linked them and harmonized endpoints to create compatible, comparison-level data suitable for correlation analysis against cancer-specific mortality.
From 1,209 candidate records identified in the reconstructed corpus, the team retrieved and assessed 996 PDFs. Thirty-three reports that were absent from the source review were prioritized by the LLM for human review. Of those 33, 26 met human-determined eligibility criteria and represented 18 separate trials. Eight of those eligible reports provided comparisons that were compatible with the original review’s comparison-level definitions and therefore could be added to the comparison set.
The original review included 41 comparisons. Adding the 8 newly identified compatible comparisons increased the dataset to 49 comparisons. The authors recalculated unweighted Pearson correlations between trial-level effects on late-stage incidence and cancer-specific mortality. The overall correlation was attenuated after adding the new comparisons: it decreased from 0.73 (95% confidence interval [CI] = 0.55 to 0.85) in the original 41-comparison set to 0.59 (95% CI = 0.37 to 0.75) in the 49-comparison set.
The audit reported updated correlations by cancer type after the additions. For breast cancer, the updated correlation was 0.49 (95% CI = −0.26 to 0.87). For colorectal cancer, the correlation became −0.23 (95% CI = −0.71 to 0.40); the authors note that a single sparse-event comparison had a meaningful influence on the colorectal estimate. For lung cancer, the updated correlation was 0.83 (95% CI = 0.54 to 0.95). These cancer-specific results demonstrate variability in stability of the association between late-stage incidence and cancer-specific mortality across tumor types.
Based on the observed sensitivity of the overall association to evidence composition and the variable cancer-specific stability, the authors recommend that late-stage incidence be evaluated by cancer type rather than assumed to be a generalizable surrogate across cancers. They advise prespecifying sensitivity analyses that address evidence selection and endpoint harmonization when late-stage incidence is proposed as a trial endpoint. Importantly, the audit underscores that model-assisted prioritization tools can help identify candidate reports but cannot substitute for human eligibility review, trial reconciliation, and source verification.
The authors report that derived audit data, the prompts used, structured model outputs, analysis code, and computational environment details are publicly available in an Open Science Framework repository (link provided in the source). The authors declared no competing interests and attested to following ethical guidelines, patient consent procedures, clinical trial registration expectations, and relevant reporting guidelines as described in the source article.