Investigators tested two available decision-only AI systems — Jev and OpenAI Decisions — against a large language model, GPT-6 Luna, using constructed clinical scenarios and published benchmarks. The cohort comprised 2,134 case presentations and involved 2,483,222 total model calls, the authors reported. The objective was narrow: compare baseline decision stability and bias susceptibility when nonclinical wording is added to otherwise identical clinical descriptions.
The study team ran controlled challenges in which prompts received alternative phrasings that did not alter clinical information but added labels or so-called pressure cues. The raw comparison permitted a head-to-head view of how each model class responded to framing differences. Investigators reported that, in these controlled tests, the decision models and the LLM both showed substantial sensitivity to wording changes — a signal that decision outputs alone are not inherently immune to framing effects.
Data indicated that adding labels or pressure cues changed decision outputs by 13.4 more decisions per 100 for the decision models when compared with neutral wording. The large language model, in isolation, changed 14.0 more decisions per 100 under the same manipulations. These figures quantify the magnitude of framing sensitivity observed in the preprint’s challenges. The authors used these differences to argue that even probability-focused decision outputs can be swayed by nonclinical text.
The study provided no additional methodological minutiae in the abstract about exact prompt wording, scoring thresholds, or the precise mapping from model responses to decision-change counts; the preprint repository and supplementary material, however, are cited as sources for constructed scenarios, prompts and mappings. The authors emphasized reproducibility by making analysis code and scenario constructs available publicly.
The core intervention tested was a paired workflow: run a decision-only model and then have the LLM evaluate or contextualize that decision. Investigators reported that pairing reduced the framing-driven decision changes substantially. In development experiments, pairing brought the observed change down to 2.6 additional decisions per 100. The team quantified relative reductions: about 81% lower sensitivity in development and between 42% and 48% lower sensitivity in held-out reports, as described in the preprint.
The authors framed this pairing as a practical proof of concept for two purposes. First, reducing the sensitivity of a decision-only output to incidental wording. Second, flagging text elements that exerted undue influence, so that human reviewers could focus attention on potentially problematic phrasing. The preprint does not claim regulatory readiness. It reports early evidence that augmenting decision outputs with LLM-driven review can reduce certain framing effects in experimental datasets.
Investigators tracked downstream measures beyond change counts. Triage accuracy for the workflows — the proportion of cases routed correctly under the study’s metrics — remained near 81% to 82% across the tested conditions. The authors reported that the paired Decisions workflow would cost at most US$0.77 per 1,000 entries in their costing estimate. That figure appeared in the abstract and serves as a ballpark operational metric for implementing a paired pipeline in low-cost settings.
The preprint does not provide, in the abstract, granular per-call latency, compute-resource breakdowns, or vendor pricing assumptions that produced the US$0.77 figure; those details are referenced as available in the public repository and supplementary files. The study team did quantify the scale of testing (millions of calls) to support the operational assertions.
The authors confirmed that the work used synthetic scenarios and publicly available benchmark data only. Sources cited include AcuityBench and several public repositories; the full constructed scenarios, prompts, model decisions, comparison mappings and analysis code were deposited at the project GitHub URL listed in the preprint. The manuscript and supplementary files, the authors said, provide the data supporting their analyses.
This is a preprint that has not undergone peer review. The authors explicitly stated that the study should not be used to guide clinical practice. The abstract does not include full protocol details, exact prompt templates, or the underlying mapping rules used to convert model outputs into the reported change-per-100 metrics; readers are directed to the supplementary materials and repository for those elements. Investigators also noted the work as an early proof of concept and presented paired workflows as a candidate mitigation strategy rather than a validated clinical solution.
Study limits recorded in the public record include reliance on synthetic or previously published benchmarks instead of live clinical deployments, and the absence of peer review at the time of posting. The authors declared no competing interests and confirmed adherence to ethical guidelines, noting that no new human subjects were recruited for these analyses. The manuscript ends on concrete next steps and open questions in the repository and supplementary material rather than broad editorial conclusions: specifically, how pairing logic generalizes across other decision models, how prompt design affects residual sensitivity in held-out data, and what human-review thresholds should be applied in operational settings.