The study team developed the ITE-Transformer, a transformer-based framework built to estimate individualized treatment effects from longitudinal electronic health records. The model pairs self-supervised representation learning of temporal diagnosis and medication trajectories with doubly robust causal machine-learning components to produce patient-level counterfactual risk estimates. Training used structured EHR data from roughly 350,000 participants in the All of Us Research Program. The source noted that participant-level data are governed by All of Us data-use policies and are not publicly shareable.
The authors described a two-stage approach. First, temporally contextualized patient representations are learned from longitudinal codes. Second, those representations feed models for treatment assignment, treatment-specific outcome risk, and derived ITEs. The abstract did not provide the exact pretraining objectives, hyperparameter values, or model-size specifications in the sections available from the source; those implementation details were not reported in the provided text.
Investigators evaluated the ITE-Transformer in two clinically distinct comparative-effectiveness settings. The first compared GLP-1 receptor agonists versus DPP-4 inhibitors in patients with type 2 diabetes; the outcome for that cohort was major adverse cardiovascular events. The second compared immunotherapy to chemotherapy in a pan-cancer cohort, where the outcome was all-cause mortality. The source presented these two use cases as demonstrations of the model across different disease domains and endpoints.
For both applications, the ITE-Transformer produced predictions of treatment assignment and of treatment-specific outcome risk, enabling estimation of counterfactual outcomes at the individual level. The paper framed the framework as a tool to ask how an individual patient’s outcome may differ under alternative treatment strategies.
Data indicated that the ITE-Transformer improved both discrimination and calibration of treatment-assignment and outcome models when compared with existing tabular machine-learning approaches. The study team also compared the full framework against an otherwise identical transformer that lacked self-supervised pretraining; the pretrained version performed better according to the abstract. The source did not include numeric performance metrics in the abstract text provided, so exact measures of improvement (for example, AUC, calibration slopes, or confidence intervals) were not reported there.
The analytic pipeline combined learned representations with doubly robust causal learners. The authors reported that this design facilitates estimation of ITEs while leveraging large-scale unlabeled temporal patterns in the EHR data. The source did not report additional methodological choices such as the specific causal estimators used, covariate adjustment strategies, or the details of any propensity-score modeling beyond the high-level description.
The ITE-Transformer identified what the authors described as substantial heterogeneity in predicted ITEs across patient subgroups. The model-generated patterns of treatment response were interpretable and associated with patients’ diagnoses, medication histories, and comorbidities. The source stated that these associations help link predicted differential benefit or harm to clinically meaningful history elements recorded in the EHR.
Investigators presented these findings as evidence that clinical foundation models can be applied to heterogeneity-of-treatment-effect questions in real-world data. The abstract did not include specific subgroup effect estimates or examples of individual-level counterfactual predictions, so readers must refer to the full manuscript for granular results and case-level illustrations.
The authors declared no competing interests. They also stated that no additional local ethical approval was necessary at LMU Munich because the analysis used de-identified, non-interventional data. The All of Us Research Program—the source of the data—had received IRB approval for the original data collection. The dataset used was the Registered Tier Dataset (version 8) on the All of Us Researcher Workbench; access is limited to authorized users under program policies.
Readers should note that the source material is a preprint posted to medRxiv. The abstract omitted several implementation specifics that are important for reproducibility, such as exact pretraining tasks, hyperparameter settings, model size, and the precise causal estimation procedures; these were not reported in the provided text. The participant-level data cannot be redistributed by the authors because of All of Us data-use restrictions. Reproducibility therefore depends on either provision of code and model artifacts by the authors or independent access to the same All of Us Researcher Workbench resources.
Limitations that remain open from the source include the absence of detailed numeric performance metrics in the abstract, lack of full disclosure of implementation parameters in the abstract text, and the preprint status of the report. Those outstanding items define the immediate next questions for readers interested in clinical validation, regulatory review, or prospective deployment of model-assisted treatment decision strategies.