The study team emulated a target trial to estimate relative vaccine effectiveness (VE) of the COVID-19 bivalent booster among Kaiser Permanente Northern California members. The investigators framed the work within cohort target-trial emulation principles, intending to preserve trial-like design while operating on observational electronic health record data. The cohort, data provenance, and exact eligibility criteria are described in the manuscript; the source did not report granular dosing schedules or detailed inclusion-exclusion lists in the abstract. The authors emphasized that emulation alone does not remove all biases, particularly those from unmeasured differences in health-seeking behavior between vaccinated and unvaccinated groups.
Data indicated residual confounding consistent with differential health-seeking behavior. The study team used negative-control analyses to probe for that residual bias. Investigators reported that even with the target-trial approach, standard adjustment failed to reconcile observed differences, motivating the use of negative controls embedded within the emulated trial framework. The authors characterized the test-negative design as a popular alternative that can reduce certain biases but observed that it often departs from strict target-trial design principles; they therefore pursued methods that operate inside the emulation paradigm while accommodating unmeasured confounding.
One approach used test negative illness as a single negative control outcome and corrected for bias under an assumption the authors called equi-confounding on the multiplicative scale. Under that assumption, the negative-control outcome provides information on the extent of unmeasured confounding and allows a multiplicative correction to the VE estimate. The source report did not provide the detailed statistical formulas or step-by-step estimation code in the abstract, but analysis code is available in the authors’ GitHub repository as noted in the manuscript. The single-negative-control method produced a 1-year relative VE estimate of 11.1% in the cohort analysis reported by the investigators.
The second approach relaxed the equi-confounding assumption but required an additional, valid negative control variable. This double negative-control strategy identifies VE without relying on equal confounding strength across outcomes; instead, it leverages the extra control to disentangle vaccine effect from unmeasured confounding. The study team observed a 1-year relative VE of 15.9% using this method. The authors specified that the double negative-control approach retains the target-trial design logic while introducing an alternative identification strategy that trades an assumption for an additional needed control measure.
Results diverged across methods. The two negative-control approaches—single and double—yielded relatively similar 1-year relative VE estimates (11.1% and 15.9%). By contrast, a traditional analysis produced a qualitatively different estimate reported as VE: -14.5%. Investigators presented these contrasts to illustrate how unmeasured confounding and method choice can materially affect VE estimates in observational vaccine studies. The manuscript puts these differences in the context of design: target-trial emulation can be maintained while accommodating negative-control strategies, and the two negative-control solutions converged toward modest positive effectiveness rather than the negative estimate from the conventional approach.
The authors noted several constraints relevant to interpretation and reuse. First, this report is a preprint and has not been certified by peer review; the manuscript itself states it should not be used to guide clinical practice. Second, the individual-level electronic health record data used in analyses contain protected health information and cannot be shared publicly because of institutional policies and data use agreements; the abstract indicates aggregate results are provided in the manuscript. Third, while analysis code is publicly available via the authors’ GitHub repository, the abstract did not include detailed parameter values, sample sizes, or p-values—readers must consult the full manuscript for those specifics. The study team also reported that Kaiser Permanente Northern California IRB approved the work and that authors declared no competing interests.
Limitations remain. The negative-control corrections rely on untestable assumptions: the single-control approach assumes equi-confounding on the multiplicative scale, and the double-control approach requires a valid additional negative control. The abstract did not present diagnostics proving the validity of the negative controls; investigators described these methods and reported VE point estimates, but the robustness of those assumptions depends on details provided in the full text. Finally, because individual-level data cannot be shared, external investigators cannot fully reproduce participant-level analyses without access through appropriate institutional agreements.
Next steps and unanswered questions specified in the report are concrete rather than speculative: peer review of the preprint, external validation of the negative-control choices, and transparent reporting of the full statistical methods in the main manuscript and accompanying code. Those steps, the authors suggest, are necessary before these methods or point estimates should influence clinical decision-making or public health guidance.