The team revisited a comprehensive benchmark that applied 2,234 harmonization approaches to a multi‑platform set of 7,174 bulk B-cell lymphoma transcriptomes. These approaches combined 31 harmonization algorithms, 14 batch-removal strategies, three imputation schemes and two post-removal settings. The investigators treated the benchmark as a single body of real-world experiments and used the registered scoring metrics from that resource—344 in total—as the raw assessment space for comparative analysis.
The authors did not report per-method hyperparameters or platform-specific preprocessing in the source; the analysis rests on the published combinations and the provided scoring outputs rather than reconstructed pipelines. This reanalysis focused on aggregate behavior across methods and metric classes rather than on tuning individual algorithms.
Rather than rely on one success metric, the study team organized the available scores into orthogonal classes to capture distinct aspects of harmonization quality. They assigned 87 polarity-adjusted scoring metrics to five algorithmic classes: global, local, distributional, structural and composite. In addition, they introduced three biology-facing classes intended to gauge biological signal preservation and cross-batch reproducibility: within-cohort marker correlation (L), cross-batch rank agreement of a marker panel (M), and leave-one-batch-out prediction of lymphoma subtypes and normal B cells (N).
For each metric class the investigators selected the top 112 approaches, representing the top 5% within that class. They then quantified overlap between those top sets using the Jaccard index, enabling a pairwise and multi-class comparison of which harmonization approaches consistently ranked highly across distinct quality definitions.
The leave-one-batch-out (LOBO) prediction class (N) specifically evaluated cross-batch classifier generalization: models trained on all but one batch and tested on the held-out batch to predict lymphoma subtypes and normal B cells. Data indicated a strong concentration of ComBat-family approaches among the best performers in this class—ComBat methods occupied 84 of the 112 top positions reported for class N.
The source did not disclose per-batch accuracy numbers or classifier types used in LOBO tests; the reported outcome is the aggregate rank representation within the top-112 list. Investigators reported that the ComBat family’s prominence in LOBO suggests robustness for cross-batch predictive tasks in this dataset, at least according to the N-class metric definition.
Approaches enforcing high distributional conformity—quantile normalization was given as an example—successfully matched batches to the same distributional shape, which can reduce obvious batch artifacts. However, such harsh distributional harmonization performed poorly on local and global metrics designed to preserve finer-grained patterns of biological variation. The study team observed that these methods can artificially align distributions while diminishing biologically relevant structure, creating a trade-off between removing batch differences and maintaining meaningful signal.
Composite and local metrics tended to favor approaches that preserved within‑ cohort structures. Local and composite metrics produced the largest top-set overlap (71 shared approaches), indicating some consensus when evaluation emphasizes preservation of localized biological relationships rather than strict distributional conformity.
Pairwise overlap between classes varied markedly. Classes L and M, and classes M and N, shared no approaches among their respective top-112 lists. Classes L and N shared a single approach, and that approach corresponded to no harmonization under a rare-batches-removed strategy. The triple overlap across L, M and N was significantly below random expectation, according to the source’s reported analysis. These patterns point to near‑mutual exclusivity among the metric definitions in terms of what they reward.
The authors also noted that the cross-batch rank agreement class (M) selected only two specific methods—10_mnn and 12_scanorama—but they flagged these methods as having produced correlation patterns consistent with biological corruption rather than faithful signal preservation according to the biology-facing evaluations.
Recognizing that the “best” harmonization depends on the metric, the investigators provided a six-step recipe aimed at guiding metric class selection, threshold setting and cross-batch testing for method choice. The source text reported the existence of this recipe and its intention to align metric selection with downstream analytical goals, but it did not enumerate the stepwise details in the body of the reanalysis summary.
Limitations remain explicit in the source: optimal harmonization is contingent on which quality metrics are used and different metric classes trade off against one another. The reanalysis did not provide per-method parameter settings, per-batch performance values, or classifier architectures used in LOBO tests, and so it cannot prescribe precise pipelines for any specific laboratory or clinical use case. The source leaves unresolved which metric thresholds should be universally adopted for particular downstream tasks, and practical selection therefore requires users to apply the team’s six-step guidance to their own data and objectives.
The dataset and code links were provided in the original resource; investigators noted these materials so users can reproduce the ranked lists and apply the recipe to new cohorts or different downstream goals. The immediate unanswered questions documented by the source are concrete: which metric class should be prioritized for a given downstream analysis, and how should users balance distributional alignment against preservation of local biological structure when choosing a harmonization approach?