Predicting clinical molecular receptor status—estrogen receptor (ER), progesterone receptor (PR), and HER2—directly from routine hematoxylin and eosin (H&E)-stained whole-slide images (WSIs) could reduce dependence on immunohistochemistry. However, purely image-based models lack direct access to the underlying transcriptional programs that define these subtypes. The authors tested whether introducing transcriptomic supervision during training would improve the capacity of a vision model to predict receptor status from WSIs alone at inference.
The approach couples a whole-slide vision transformer (GigaPath) with bulk RNA-sequencing profiles through a contrastive objective applied only at training time. This design enables the slide encoder to learn representations aligned with transcriptomic signals while preserving the ability to make predictions from histology images alone during deployment. The contrastive loss enforces similarity between paired histology and transcriptomic representations of the same patient and dissimilarity across patients.
Bulk RNA-seq profiles were drawn from The Cancer Genome Atlas breast cancer cohort (TCGA-BRCA). Rather than training a bespoke bulk-transcriptomics encoder, the authors repurposed a gene-sentence encoding and a pretrained text encoder from BioBERT to represent transcriptomic data for the contrastive objective. This leverages pretrained language-model representations for gene-level expression patterns and integrates them with the slide-level visual encoder during contrastive pretraining.
Under five-fold cross-validation on TCGA-BRCA, RNA-guided contrastive pretraining consistently improved AUROC for ER, PR, and HER2 relative to a baseline that lacked contrastive pretraining. Beyond AUROC gains, the RNA-guided approach reproducibly introduced receptor-relevant structure into the learned slide representations, indicating that the model captured transcriptionally meaningful features from histology.
When evaluated on independent external cohorts, results diverged in a receptor-dependent manner. For ER and PR—receptors governed by broad, multi-gene luminal transcriptional programs—the performance gains generalized robustly to external datasets. In contrast, the apparent AUROC improvement for HER2 did not translate cleanly: HER2 prediction is driven by focal ERBB2 amplification rather than a coordinated transcriptional signature, and on external cohorts the AUROC increase masked a substantial loss of classification sensitivity at the chosen operating threshold. In other words, the training-time transcriptomic supervision produced different real-world implications depending on the molecular basis of the target label.
These results indicate that training-time transcriptomic supervision is a practical route to enhancing histology-only molecular subtyping, but its benefits depend on the biological nature of the molecular alteration and on gene-selection strategy. Supervision derived from broad, coordinated transcriptional programs (for example, the luminal programs associated with ER and PR status) can teach a vision model to recognize histologic correlates that generalize across cohorts. By contrast, alterations that are focal at the genomic level and lack a broad transcriptional footprint (for example, ERBB2 amplification driving HER2 status) may be less amenable to robust learning through transcriptomic contrastive supervision; apparent improvements in AUROC may obscure clinically important losses in sensitivity.
The method allows deployment of histology-only inference while leveraging paired RNA-seq during training, which may reduce the need for routine immunohistochemistry in settings where the learned histology signals generalize. The findings also emphasize that gene-selection strategy—which genes or transcriptomic features are encoded and used for supervision—and the underlying biology of the molecular target determine whether transcriptomic guidance will be effective. Because the report is a preprint, the work has not been peer reviewed. Details beyond what is reported in the abstract (for example, specific AUROC values, sample sizes, preprocessing steps, threshold choices, and the composition of external cohorts) were not reported in the abstract and are not recapitulated here.
RNA-guided contrastive learning that aligns vision transformer representations with bulk transcriptomic encodings can improve receptor-status prediction from H&E WSIs and introduce biologically meaningful structure into slide embeddings. However, generalization depends on whether the molecular target corresponds to a broad transcriptional program or a focal genomic event, highlighting the need for careful gene-feature selection and external validation when applying transcriptomic supervision to histology-based molecular subtyping.