This study describes a knowledge distillation pipeline designed to transfer multimorbidity reasoning from large language model (LLM) teacher models into smaller, deployable student models referred to as CoLLMs. The motivation is to retain the rich clinical knowledge encoded by contemporary LLMs while avoiding direct exposure of sensitive patient records when evaluating or scaling these models on large biobank datasets.
The framework emphasizes privacy by avoiding direct use of real patient-level data during the distillation step. Instead, the authors create an intermediary synthetic dataset that reflects distributional characteristics of the target cohort. Distillation then proceeds with teacher LLMs providing multimorbidity reasoning and labels that the student models learn to approximate.
The goal is twofold: (1) achieve high-fidelity transfer of clinically meaningful multimorbidity signals from large LLM teachers to compact students, and (2) enable analyses and downstream use on real-world biobank data while maintaining patient privacy.
A central element of the approach is the generation of synthetic cohorts that preserve statistical properties of the UK Biobank sample without exposing individual-level records. These synthetic cohorts are used as the medium for distillation, so teacher models can be queried and the resulting outputs used to train CoLLMs off-line.
By preserving UK Biobank distributions in synthetic data, the method aims to keep downstream performance and generalizability aligned with the target population while reducing the risk of patient data leakage from teacher model interactions. The paper reports that this strategy enables large-scale LLM-informed modeling while maintaining a privacy-aware workflow.
The authors quantify knowledge transfer fidelity between teacher LLMs and distilled CoLLMs using rank-correlation metrics. Reported Spearman rho values for the fidelity of multimorbidity reasoning range from 0.75 to 0.89, indicating substantial but not perfect agreement between teachers and students across evaluated settings.
The analysis also highlights substantial variability among teacher models in the content and strength of the multimorbidity reasoning they provide. This variability was detectable both in direct fidelity metrics and in downstream evaluations, underscoring that teacher selection materially influences the distilled student behavior.
To assess clinical significance beyond numeric fidelity metrics, the study uses an independent LLM-as-a-Judge evaluation. In this setup, a separate LLM evaluates the clinical relevance of the distilled multimorbidity outputs.
This independent judgment confirmed that distilled knowledge retained clinical meaning and utility. The LLM-as-a-Judge evaluation also served to surface discrepancies and variability across teacher models, supporting the finding that teacher heterogeneity affects the distilled outputs.
CoLLM-derived multimorbidity scores were applied to real UK Biobank data to evaluate clinical utility. When used for survival prediction, the distilled scores improved model performance, achieving a reported C-index up to 0.91 in the analyses presented.
Genetic signal in the CoLLM-derived scores was evaluated via SNP heritability. The authors report higher SNP heritability for the CoLLM scores, with h2 approximately 0.05, suggesting that the distilled multimorbidity constructs capture measurable genetic contribution consistent with phenotypic relevance.
Together, these downstream results indicate that privacy-aware distillation can produce compact models whose outputs are both clinically informative and genetically sensible when applied to a large biobank cohort.
The authors make secondary data, code, and a web application available: the project repository is hosted at https://github.com/rxa615/CoLLM and the CoLLM web application is accessible at https://zhulab-collm.streamlit.app/. Primary data used for downstream application originate from the UK Biobank and may be obtained through that resource.
Ethical oversight for use of the UK Biobank resource was provided by the North West-Haydock Research Ethics Committee of the UK Health Research Authority (REC reference 21/NW/0157). All UK Biobank participants provided informed consent. The authors declare no competing interests.
Funding was declared from the National Human Genome Research Institute (NHGRI) under award identifiers reported in the source.
The presented approach offers a structured path to leverage strengths of large LLM teachers while minimizing privacy risks that accompany direct teacher access to sensitive clinical records. By operating through synthetic cohorts that mirror target distributions, the method allows distilled student models to be trained and then applied on real cohorts such as the UK Biobank.
Empirical findings in this report—high Spearman rho values for fidelity, LLM-judge confirmation of clinical relevance, improved survival prediction (C-index up to 0.91), and measurable SNP heritability (h2 approximately 0.05)—support the feasibility of this privacy-aware pipeline. The study also highlights that choice of teacher LLMs affects outcomes, suggesting careful selection and evaluation of teacher models are important for trustworthy deployment.
Limitations and details not reported in the source: the preprint text provides aggregate fidelity and downstream metrics but does not enumerate exact teacher model names, all hyperparameter settings for synthetic data generation, or full distributional diagnostics in this summary; readers should consult the linked repository and supplementary material for methodological specifics.
Overall, the work establishes a reproducible, privacy-conscious strategy for bringing LLM-derived clinical reasoning into scalable multimorbidity modeling and downstream genomic and survival analyses.