Falls and fall-related injuries among older adults are a major clinical and public health concern. The authors aimed to develop and validate a supervised text-embedded transformer matching model to detect fall injuries in Medicare claims and encounters. The primary objective was twofold: (1) to create a model that produces a continuous probability that a claims record represents a fall injury, and (2) to compare its performance against a validated rule-based algorithm using an external reference standard of self-reported fall injuries that led to medical attention.
Inputs to the model were drawn from Medicare claims and encounter data for participants enrolled in the Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) trial. The model used text embeddings derived from diagnostic and procedure coding present in claims: ICD-10-CM and CPT codes. These embeddings represented each claim/encounter as input tokens for the transformer-based architecture.
The external reference standard for evaluating model performance consisted of participant self-reports of fall injuries that required medical attention, as captured in STRIDE. Annotated claims/encounters that occurred within ± one month of those self-reported events were used for model training and internal validation.
A supervised transformer matching model was trained on annotated claims/encounters linked temporally to self-reported fall injuries. Training labels derived from the annotation process allowed the model to learn patterns in the embedded representations of ICD-10-CM and CPT codes associated with fall-related care. The model produced a continuous output on a 0–1 scale representing the predicted probability that a specific claim or encounter corresponded to a fall injury.
The choice to generate a probability rather than a binary classification was intentional, with the authors noting potential utility for downstream processes such as endpoint adjudication in trials or surveillance.
Internal cross-validation on annotated claims/encounters used a 10-fold holdout approach. Against these annotated data, the transformer model attained an area under the receiver operating characteristic curve (AUC) greater than 0.96 in nine of ten folds and an AUC of 0.85 in the remaining fold.
When applied to the full STRIDE dataset and evaluated against the external reference standard of self-reported fall injuries leading to medical attention, the model achieved a peak AUC of 0.86 (95% CI, 0.84–0.87). These results indicate good discrimination between fall-related and non–fall-related claims in this external validation context.
The transformer model's performance in the full STRIDE dataset was described as comparable to that of a previously validated rule-based algorithm. Rule-based methods typically yield binary outcomes (event/non-event). By contrast, the transformer model returns a continuous event probability which the authors argue may offer operational advantages for clinical endpoint workflows.
No numerical head-to-head performance metrics or detailed operating point comparisons were reported in the source beyond the statement of comparability and the AUC values against the external reference standard.
The continuous probability output from the transformer model supports a tiered adjudication strategy: high-probability predictions can be treated as events, moderate-probability predictions can be routed for manual adjudication, and low-probability predictions can be considered non-events. The authors present this as a potential improvement over rigid binary outputs from rule-based algorithms when applied to event ascertainment in trials or observational studies.
The source frames the work as a “proof of concept” for the use of text-embedded transformer outputs in clinical endpoint adjudication, rather than as a final, deployable adjudication system.
The study received institutional review approval from Massachusetts General Brigham IRB as an amendment to protocol 2015P000693. The authors reported that patient consent and necessary institutional approvals were obtained and that they followed relevant reporting guidelines. No competing interests were declared.
Medicare data used in the study cannot be shared with third parties because of the Centers for Medicare and Medicaid Services data use agreement. A deidentified version of the original STRIDE study data can be requested from the Aging Research Biobank.
Funding was provided by the National Institute on Aging, the National Center for Advancing Translational Sciences, and the Patient-Centered Outcomes Research Institute, with grant/support details reported in the source.
A supervised text-embedded transformer model using ICD-10-CM and CPT code embeddings identified fall injuries in Medicare claims with discrimination comparable to a validated rule-based algorithm when measured against self-reported fall injuries in STRIDE. The model produced continuous event probabilities (peak AUC 0.86 against the external standard), which the authors suggest could enhance endpoint adjudication workflows by triaging claims for manual review. The study is presented as proof of concept; further work would be required to evaluate deployment, operational thresholds, and generalizability beyond the STRIDE-linked Medicare population.