Cross-modal fusion of cytomorphology and 18F-FDG PET/CT for non-invasive bone marrow immune microenvironment decoding in multiple myeloma

IntroductionThe bone marrow immune microenvironment (BMME) shapes treatment response in multiple myeloma (MM), yet routine diagnostic workup primarily assesses tumor burden rather than immune competence. We developed ImmunoCast-MM, a cross-modal deep learning framework that extracts immunologically relevant signals from two examinations routinely performed at diagnosis: Wright–Giemsa-stained bone marrow aspirate smears and whole-body 18F-FDG PET/CT.MethodsThe cytomorphology branch used DinoBloom embeddings to classify individual cells across five hierarchical levels.

The PET/CT branch generated a multi-organ inflammation fingerprint from tumor, spleen, lymph node, and diffuse bone marrow compartments. A contrastive fusion module aligned the two imaging modalities with a flow cytometry reference panel and generated an Immune Dysfunction Index (IDI) along a learned effector–suppressor axis.

ImmunoCast-MM was evaluated retrospectively in 243 patients with newly diagnosed MM. Associations with flow cytometric measurements, progression-free survival, and daratumumab response were assessed, with adjustment for International Staging System stage, cytogenetic risk, and age.ResultsOn a held-out validation subset, decoded cytomorphologic indices correlated with matched flow cytometric fractions, with Spearman ρ values of 0.68–0.81 across three decoded index–panel pairs; all Benjamini–Hochberg-adjusted (p) values were (<0.001).