Offline reinforcement learning (RL) has been proposed as a framework for clinical decision support in sepsis, but most prior work has focused on adult populations. The authors highlight important physiological and treatment differences in children that warrant dedicated pediatric investigation. This study aims to build and evaluate offline RL policies for pediatric sepsis management in the Pediatric Intensive Care Unit (PICU) using retrospective clinical data.
The dataset comprises 2,229 episodes from Great Ormond Street Hospital (GOSH). The clinical task is formalized as a finite‑horizon Markov Decision Process (MDP) with joint action choices for intravenous fluids and vasopressors. The terminal outcome used in some reward formulations is 90‑day mortality. To better capture pediatric organ dysfunction dynamics, the authors also incorporate Phoenix 8, a recently proposed pediatric sepsis severity score, as an intermediate shaping reward in some experiments.
Joint action space: simultaneous decisions about fluid and vasopressor dosing are represented in each MDP time step. The authors systematically vary the time‑step size to evaluate temporal granularity: 4, 8, and 12 hours per bin. Reward structures tested include terminal 90‑day mortality alone and terminal mortality augmented with intermediate Phoenix 8‑based shaping signals. The study examines how these design choices affect learning stability and clinical plausibility of policies.
Three modeling approaches are compared:
These models are evaluated across the different time bins and reward formulations described above.
Model assessment uses internal learning dynamics and Fitted Q Evaluation estimates reported by the authors. Key empirical findings reported in the source are:
Temporal aggregation (increasing time‑step size) functions as a regularizer in these experiments. Moving from 4‑hour to 8‑hour bins reduces horizon length and smooths reward noise, which the authors report improves training stability without eliminating clinically meaningful dynamics. The 8‑hour binning is identified as the best trade‑off between stability and granularity in this dataset. Twelve‑hour bins further reduce noise but at the cost of coarser temporal resolution.
CQL policies achieved high action‑level agreement with historical clinician decisions for both fluids and vasopressors. They also reproduced clinically plausible escalation patterns across sepsis severity strata as characterized in the cohort. In contrast, the DDQN policies diverged more frequently toward dosing decisions the authors deemed implausible in this setting.
The findings highlight two practical design considerations for offline RL in healthcare:
The authors argue these design choices can enhance robustness of offline RL for pediatric sepsis and suggest alternatives to the conventional 4‑hour setup deserve consideration.
The work is reported as having ethical oversight from the National Health Service (NHS) Health Research Authority (IRAS ID: 301663, REC reference 21/LO/0646). The authors declare no competing interests. Funding for the study was provided by Great Ormond Street Hospital Children's Charity (VS0618). The manuscript is a preprint and has not been peer reviewed; the authors note it should not be used to guide clinical practice.
The source article is a preprint (not peer reviewed). Specific limitations beyond those inherent in offline RL and retrospective cohort analyses are not enumerated in detail in the provided text. The article emphasizes empirical findings within the GOSH cohort and does not claim external validation across other centers in the presented excerpt.