Investigators reported that a single AI tool was introduced to auto-categorize and auto-route every incoming patient portal message to the neurology In-Basket across two clinical sites within a regional system. The intervention period ran from January to May 2026 and targeted operational efficiency: the tool sorted messages and produced optional draft replies, while clinical staff retained final responsibility for responses. The cohort analyzed comprised 24,142 messages; that count and other descriptive data were provided by the study team.
Before deployment, median weekly time to first action varied between sites: Site A showed a median of 11.3 hours (IQR 5.6–15.1), while Site B’s median was 16.1 hours (IQR 13.8–19.0). After the tool went live, medians dropped to 1.0 hour (IQR 0–3.1) at Site A and 0.6 hour (IQR 0.2–2.4) at Site B. Those changes in initial handling were large and immediate. Authors noted that the objective process measures were time from message creation to first action and time from creation to closure; no additional clinical outcome measures were reported in the source.
The study team analyzed impact using a controlled interrupted time-series (ITS) design, comparing the neurology divisions implementing the tool with message-volume-matched control divisions. At Site A, ITS showed a significant immediate differential level change in median weekly hours to first action (reported as -92.5%, 95% CI -97.5% to -78.0%). At Site B, investigators observed a statistically significant downward trend in the ITS slope for time to first action (reported as -43.5%, 95% CI -65.2% to -8.2%).
Data indicated that reductions in time to closure followed the same direction as time to first action, but those changes did not reach statistical significance at either site according to the authors. The source did not provide further subgroup analyses, operational cost metrics, or breakdowns by message type beyond aggregate process outcomes. Investigators therefore reported process improvements without detailing downstream clinical effects or patient-centered endpoints.
The AI produced optional draft replies for clinicians to use. Usage, however, was very low. The study team reported that generation-to-use rates were 0.5% at Site A and 3% at Site B. The authors did not report qualitative data explaining this low adoption—such as clinician attitudes, workflow friction, or draft quality scores—so the reasons for minimal uptake remain unspecified in the source. Clinicians retained ultimate control over messaging decisions, and the tool functioned as an assistive, not autonomous, system.
Low draft-reply use is an operational signal worth attention. It suggests that auto-routing and categorization may deliver the primary efficiency gains seen in this implementation, while automated content generation did not substantially alter clinician behavior during the study period. The manuscript provides counts and percentages but does not supply examples of draft replies, user feedback metrics, or safety audits for generated content.
The authors stated that the study received approval from the Mass General Brigham Institutional Review Board, with waiver of informed consent because analyses used de-identified data collected as part of routine care. The source explicitly notes that this report is a preprint and has not been peer reviewed; investigators cautioned that findings should not yet guide clinical practice. The article lists the Baker Family Initiative in Patient Experience and Outcomes as a funder and indicates anonymized data are available on request.
Methodological limitations were implicit in the source. The ITS design used control divisions matched on message volume, but the manuscript did not report whether other potential confounders were adjusted for, nor did it provide granular data on message content categories or clinician-level workloads. Authors reported significant reductions in the initial response interval, but they did not demonstrate statistically significant changes in time to closure, and they did not present patient outcome measures tied to faster responses.
Open questions remain and are documented in the source: why clinicians rarely used draft replies; whether reduced time to first action translates into measurable patient outcomes; how the tool performs across different service lines or message types; and what governance or validation processes were applied to the AI’s categorization rules. These items were not answered in the preprint and will require further evaluation and peer review.