Reliable deployment of automated liver CT segmentation requires mechanisms to detect failures that occur in rare or previously unseen situations. The authors build on prior work using Pairwise Surface DSC as an indicator of segmentation quality. To approximate the in-distribution behavior of this quality metric, they fitted a log-t distribution to Pairwise Surface DSC scores derived from a validation set of 400 internal scans. This fitted distribution serves as the reference for assessing how extreme new segmentation scores are relative to the expected in-distribution variation.
For each new segmentation, the method computes a significance score representing how extreme the Pairwise Surface DSC is under the fitted log-t distribution. Using statistically principled cutoffs, segmentations are assigned to one of three review categories: Low Risk, Medium Risk, and High Risk. The specific significance thresholds reported are 0.25 and 0.05, which define the boundaries between Low/Medium and Medium/High Risk respectively. These thresholds convert a continuous extremity measure into actionable review categories without requiring labeled failure examples.
The authors report that the fitted log-t distribution provided a strong fit to the observed Pairwise Surface DSC scores from the validation cohort. They also state that the fitted model remained robust when the validation set contained a moderate amount of OOD contamination, indicating the approach can tolerate some degree of unlabeled failures while still approximating the in-distribution score distribution. Details on the quantitative fit metrics or contamination levels were not reported in the abstract.
Performance of the label-free thresholding framework was evaluated on an independent test set of 500 scans that included both internal and external cases. Using the defined review categories, the authors report two key operating points:
Combining the Medium and High Risk categories (i.e., marking anything not Low Risk for review) achieved 100% sensitivity and 79% specificity for detecting segmentation failures on the test set.
Using the High Risk category alone (i.e., a more conservative alert threshold) yielded 78% sensitivity and 96% specificity.
These results illustrate a trade-off between sensitivity and specificity driven by the chosen review threshold: a broader review policy (Medium + High) increases sensitivity to capture all failures, while a narrower High Risk-only policy reduces false positives at the cost of missing some failures.
A central motivation for the approach is to reduce the annotation burden inherent in threshold calibration when failures are rare. Traditional thresholding requires expert-labeled failures to tune operating points. By fitting an in-distribution model to unlabeled validation scores and deriving significance-based cutoffs, the proposed framework enables threshold calibration from unlabeled data. The reported test-set results suggest that clinically meaningful failure detection can be achieved without assembling a labeled failure set, which could simplify system validation and monitoring in real-world deployments.
The underlying retrospective study received approval from The University of Texas MD Anderson Cancer Center Institutional Review Board (PA18-0832); the requirement for written informed consent was waived for this retrospective analysis. The authors note that the MD Anderson data used in the study may be made available upon request in compliance with institutional IRB requirements and MD Anderson policies.
The authors have made their code available at the GitHub repository: https://github.com/marshalln7/Label_Free_OOD_Threshold_Selection. They declare no competing interests.
Funding sources disclosed include the MD Anderson Tumor Measurement Initiative through the STRIDE program and multiple National Cancer Institute grants (P30CA016672, 1R01CA221971, R01CA235564, P01CA261669).
This work is reported as a medRxiv preprint and has not undergone peer review; the authors explicitly note that the findings should not be used to guide clinical practice. The abstract does not report detailed quantitative fit statistics for the log-t model, the exact composition of the external cases in the test set, or the precise definition and count of segmentation failures used for performance evaluation beyond the sensitivity and specificity values provided. Those details may be present in the full manuscript or supplementary materials but were not reported in the abstract.