General-purpose large language models are increasingly used by clinicians and triage staff to support emergency triage decisions. Prior work cited by the authors reported high rates of undertriage when patients queried a general-purpose model directly. The authors developed MTS-Bench to assess triage safety by mapping published clinical scenarios to the Manchester Triage System (MTS) and measuring how different language model configurations perform against this structured, widely used triage framework.
The benchmark adapted the clinical scenarios published by Ramaswamy et al. and mapped each scenario to an appropriate MTS priority, resulting in 39 emergency cases that covered all five MTS priority levels. Each case was evaluated in two input variants: one without objective clinical data and one with an objective clinical data block containing vital signs, examination findings, and laboratory results. Cases were permuted across two genders, producing 156 distinct prompts per condition. An anchoring manipulation — a misleading GP referral statement — was prepended for half the prompts, doubling the total to 312 prompts per system configuration.
Three model configurations were tested:
Each system received the full set of prompts, with and without the anchoring GP referral statement, and with and without objective clinical data.
The primary outcome was the undertriage rate measured on the ordered MTS scale. The study also recorded occurrences of dangerous misses (cases where a high-acuity presentation was missed) and examined the effect of the anchoring statement and of adding objective clinical data on each system's triage assignments. Statistical comparisons used Fisher's exact test where reported.
GPT-5.1 undertriaged 44.2% of cases (69/156; 95% CI 36.7 to 52.1). High-acuity presentations were frequently undertriaged by GPT-5.1: 75.0% of Red and 73.4% of Orange presentations were undertriaged.
Both DR. INFO configurations undertriaged 11.5% of cases (18/156; 95% CI 7.4 to 17.5). The difference in undertriage between each DR. INFO configuration and GPT-5.1 was statistically significant (Fisher's exact p = 1.0 x 10^-10).
GPT-5.1 produced six dangerous misses (3.8%), whereas both DR. INFO configurations produced zero dangerous misses; this difference was statistically significant (p = 0.030).
There was no significant effect of patient gender on triage performance across the evaluated systems.
When the misleading GP referral anchoring statement was prepended, GPT-5.1 undertriaged all Red cases in that anchored subset (8 of 8). Both DR. INFO configurations continued to undertriage none of the Red cases with the anchoring statement.
Adding objective clinical data to the input markedly reduced undertriage for DR. INFO with MTS retrieval: undertriage fell from 19.2% to 3.8% (p = 0.005). DR. INFO Baseline and GPT-5.1 did not show a comparable improvement when objective data were added.
The primary comparisons reported used Fisher's exact test. The undertriage reduction for DR. INFO configurations compared with GPT-5.1 reached a highly significant p-value (1.0 x 10^-10). The absence of dangerous misses for both DR. INFO configurations versus six for GPT-5.1 was reported with p = 0.030. The reduction in undertriage for DR. INFO with MTS retrieval when objective data were added was p = 0.005. Confidence intervals are reported for main proportions where provided by the source.
The authors state that MTS-Bench is available from the corresponding author on reasonable request. The clinical scenarios adapted for MTS-Bench were published by Ramaswamy et al. (2026). The Manchester Triage Group reference text used as the gold standard is Mackway-Jones et al. (2014, John Wiley & Sons/BMJ Books), which the authors retrieved for the MTS-specific configuration.
On this benchmark, the agentic retrieval-augmented system (DR. INFO) — both baseline and with explicit MTS textbook retrieval — substantially reduced undertriage rates and eliminated recorded dangerous misses compared with a general-purpose model (GPT-5.1). Retrieving the Manchester Triage System textbook at inference time was associated with greater resistance to a misleading anchoring statement and with an appropriate change in assigned MTS priority when objective clinical data arrived. Among the three evaluated configurations, only DR. INFO with MTS retrieval combined an initially conservative triage assignment with appropriate updating as additional clinical information was provided.
Notes and limitations drawn from the source
This report is a preprint and has not been peer reviewed; the authors explicitly note it should not be used to guide clinical practice. Full access to MTS-Bench is provided on request from the corresponding author. The source provides the specific numerical results summarized above; no additional outcome data beyond what the authors reported were available in the source.