Investigators evaluated APOLLAI, a multimodal AI system, against emergency medical services (EMS) personnel using recordings from 295 patients with suspected stroke. The system produced higher sensitivity than EMS for each of the five examined neurological tasks — best gaze, facial palsy, motor arm, aphasia and dysarthria — and for the composite, global assessment (reported sensitivities: 78.5% for APOLLAI versus 59.4% for EMS for the global assessment). Specificity was similar between systems and clinicians (global specificity reported as 87.0% for APOLLAI and 85.9% for EMS). Area under the ROC curves for APOLLAI ranged from 0.839 for facial palsy to 0.966 for best gaze, with a global AUC of 0.882, indicating strong discriminative performance on the study recordings.
Short, concrete numbers appear in the source. The study team reported those performance metrics directly; no additional subgroup breakdowns or alternative thresholds were provided in the preprint.
Seven certified stroke neurologists independently annotated the five NIH Stroke Scale items on each recording. The authors used the Dawid-Skene algorithm to merge independent annotations into consensus labels that served as the reference standard for performance comparison. Interrater agreement among these neurologists was reported as moderate overall (mean κ = 0.541, range 0.367–0.732), a metric the investigators presented to characterize intrinsic assessment difficulty and variability among experts.
The EMS assessments were collected separately: the source reported a mean of 15 EMS raters per patient. The study team compared both EMS-derived labels and APOLLAI outputs against the neurologist consensus to determine sensitivity, specificity and AUC values.
The preprint describes APOLLAI as a multimodal machine-learning framework that combines computer vision and audio-based deep learning models, with a final support vector machine (SVM) fusion classifier. Video input targeted visible motor and gaze features; audio streams were used to assess speech elements relevant to aphasia and dysarthria. The SVM fusion stage aggregated modality-specific predictions into item-level outputs and a global assessment.
The source did not provide implementation-level details such as model architectures, training hyperparameters, compute requirements, inference latency, or how the system would be deployed in real-time EMS workflows. Those operational and engineering details were not reported in the preprint.
APOLLAI outperformed EMS personnel on sensitivity for every task examined in the study; the global sensitivity difference reported was substantial (78.5% versus 59.4%). Specificity was broadly comparable across the two approaches for the global assessment (87.0% versus 85.9%). AUC values for individual items also favored APOLLAI, with a low near 0.839 for facial palsy and a high near 0.966 for best gaze. The global AUC was 0.882.
The authors used these diagnostic-performance metrics to argue that APOLLAI may offer a standardized approach that increases case detection without sacrificing specificity. The preprint does not report statistical testing details beyond the point estimates cited in the abstract, and it does not present subgroup analyses by age, symptom duration, or stroke subtype within the public text.
The seven neurologists showed moderate agreement (mean κ=0.541), a result the study team reported to underline subjectivity and variability in neurological assessments—even among certified experts. That variability informed the decision to use algorithmic consensus labeling (Dawid-Skene) rather than a single rater as ground truth.
The authors noted that this intrinsic variability complicates any comparison between automated systems and human raters, because the reference standard itself reflects some level of disagreement. The preprint did not include detailed analyses of which specific items drove the largest neurologist disagreements beyond the κ range provided.
Ethics approvals were reported: the protocol received clearance from relevant French committees (CESREES and Comité de Protection des Personnes Nord Ouest IV) and the French data-protection authority (CNIL), and the authors stated that informed consent procedures were observed. The dataset analyzed in the study is not publicly available due to the presence of sensitive patient data and ethics/regulatory restrictions.
The preprint lists several limitations. It is a single-center prospective study using recordings obtained at Nîmes University Hospital between August 2023 and July 2025; external validation on independent cohorts and deployment in operational EMS settings were not reported. The authors called attention to the moderate interrater agreement among experts and to the need to test the system in real-world prehospital conditions before drawing operational conclusions.
The source concludes that APOLLAI’s higher sensitivity, with preserved specificity on recorded data, supports further evaluation as a scalable adjunct to prehospital stroke recognition. The study team recommended prospective validation under real-world EMS conditions. The preprint did not report details on how APOLLAI would be integrated into ambulance workflows, whether real-time inference was feasible on standard EMS equipment, what training or interpretability tools would accompany the model, or how the system performs on data from other centers or varied patient populations.
Those open questions—external validation, real-time deployment logistics, inference latency, and generalizability across different EMS systems and recording conditions—remain the next concrete steps documented by the authors for future work.