--- title: "Accuracy of EMR Retrieval Methods and a Large Language Model for Identifying Cardiovascular Events" id: "bmj-open-6-diagnostic-accuracy-of-electronic-medical-record-retrieval-methods-and-a-large" canonical_url: "https://medichelpline.com/clinical-feed/bmj-open-6-diagnostic-accuracy-of-electronic-medical-record-retrieval-methods-and-a-large" content_type: "clinical_feed_article" specialty: "Cardiology" source_name: "BMJ Open" source_url: "http://bmjopen.bmj.com/cgi/content/short/16/8/e116133?rss=1" published_at: "2026-08-13T09:40:06.000Z" evidence_level: "Journal Feed" license: "CC-BY-NC-4.0 / Informational Use" --- # Accuracy of EMR Retrieval Methods and a Large Language Model for Identifying Cardiovascular Events ## Provenance & Clinical Metadata - **Canonical URL:** https://medichelpline.com/clinical-feed/bmj-open-6-diagnostic-accuracy-of-electronic-medical-record-retrieval-methods-and-a-large - **Specialty:** [Cardiology](https://medichelpline.com/clinical-feed/cardiology.md) - **Primary Source:** BMJ Open - **Source URL:** [Original Journal Publication](http://bmjopen.bmj.com/cgi/content/short/16/8/e116133?rss=1) - **Published At:** 2026-08-13T09:40:06.000Z - **Evidence Rating:** Journal Feed ## Executive GIST (TL;DR) - Objective: Compare diagnostic accuracy of four automated **electronic medical record (EMR)** retrieval methods, including a **large language model (LLM)** workflow, against manual chart adjudication for cardiovascular events. - Design and setting: Retrospective diagnostic accuracy study across three sites within a single US tertiary health system. - Participants: Two adult cohorts with prior adjudicated cardiovascular outcomes: Cohort 1 (n=2258) treated with immune checkpoint inhibitors; Cohort 2 (n=1426) who underwent transcatheter aortic valve replacement. - Outcomes: Reference standard was clinician manual chart adjudication. Evaluated events were ischaemic **stroke** or transient ischaemic attack, **myocardial infarction (MI)**, **heart failure (HF)** exacerbation or hospitalization, and composite **major adverse cardiovascular events (MACE)**. - Automated methods compared: International Classification of Diseases (**ICD**) codes, primary diagnosis, problem list, and a zero-shot **LLM** workflow. - Metrics: Area under the receiver operating characteristic curve (**AUC**), sensitivity, specificity, and net reclassification improvement. - Key results Cohort 1: LLM had highest AUC for stroke (0.920; 95% CI 0.881–0.958), MI (0.938; 95% CI 0.905–0.971), and MACE (0.880; 95% CI 0.854–0.907). **ICD** retrieval had higher AUC for HF (0.882; 95% CI 0.845–0.918) than LLM (0.873; 95% CI 0.831–0.914). - Key results Cohort 2: LLM achieved the highest AUC across all outcomes: stroke (0.915; 95% CI 0.862–0.968), MI (0.928; 95% CI 0.839–1.000), HF (0.844; 95% CI 0.803–0.884), and MACE (0.862; 95% CI 0.829–0.895). - Statistical comparisons: In Cohort 1, AUC differences between LLM and ICD were not statistically significant for evaluated outcomes; in Cohort 2, LLM showed significantly higher AUC for stroke and composite MACE. - Conclusion: The **LLM-assisted workflow** demonstrated strong but context-dependent performance. Performance varied by outcome and cohort, and **ICD**-based retrieval remained competitive for some use cases. Findings support a complementary role for LLM-assisted extraction in retrospective cardiovascular outcomes research. ## Clinical Analysis & Structured Key Points Objective To compare the diagnostic accuracy of four available automated electronic medical record (EMR) retrieval methods, including a large language model (LLM)-assisted workflow, against manual chart adjudication for identifying cardiovascular events. Design Retrospective diagnostic accuracy study. Setting Three sites within a single US tertiary health system. Participants Two adult cohorts with previously adjudicated cardiovascular outcomes were included. Cohort 1 included 2258 patients treated with immune checkpoint inhibitors, and Cohort 2 included 1426 patients who underwent transcatheter aortic valve replacement. Primary and secondary outcome measures The reference standard was clinician manual chart adjudication. Outcomes included ischaemic stroke or transient ischaemic attack, myocardial infarction (MI), heart failure (HF) exacerbation or hospitalisation and a composite major adverse cardiovascular events (MACE) outcome. Automated retrieval methods included International Classification of Diseases (ICD) codes, primary diagnosis, problem list and a zero-shot LLM workflow. Area under the (receiver operating characteristic) curve (AUC), sensitivity, specificity and net reclassification improvement were assessed. Results In Cohort 1, the LLM achieved the highest AUC for stroke (0.920; 95% CI 0.881 to 0.958), MI (0.938; 95% CI 0.905 to 0.971) and composite MACE (0.880; 95% CI 0.854 to 0.907), whereas ICD-based retrieval had a higher AUC for HF (0.882; 95% CI 0.845 to 0.918 vs 0.873; 95% CI 0.831 to 0.914). In Cohort 2, the LLM achieved the highest AUC for all evaluated outcomes: stroke (0.915; 95% CI 0.862 to 0.968), MI (0.928; 95% CI 0.839 to 1.000), HF (0.844; 95% CI 0.803 to 0.884) and composite MACE (0.862; 95% CI 0.829 to 0.895). In Cohort 1, differences in AUC between the LLM and ICD methods were not statistically significant across outcomes, whereas in Cohort 2 the LLM showed significantly higher AUC for stroke and composite MACE. Conclusion In this multisite retrospective validation study, the LLM-assisted workflow showed strong but context-dependent performance for identifying cardiovascular events from the EMR. Performance varied by outcome and cohort, and ICD-based retrieval remained competitive for some use cases. These findings support a complementary role for LLM-assisted extraction in retrospective cardiovascular outcomes research. ## Related Clinical Research - [Long-term cardiovascular risk after assisted reproductive technology and infertility: nationwide c](https://medichelpline.com/clinical-feed/plos-medicine-0-long-term-risk-of-cardiovascular-disease-after-assisted-reproductive-technology.md) - [Intracoronary prourokinase for slow‑flow/no‑reflow during PCI in ACS: randomized double‑blind tria](https://medichelpline.com/clinical-feed/bmj-open-5-efficacy-and-safety-of-intracoronary-injection-of-recombinant-human.md) - [Recombinant shingles vaccine associated with reduced risk of cardiovascular events in older adults](https://medichelpline.com/clinical-feed/nature-2-recombinant-shingles-vaccination-and-the-risk-of-cardiovascular-events.md) - [Cardiometabolic pathways mediating the effect of educational attainment on cardiovascular disease:](https://medichelpline.com/clinical-feed/medrxiv-13-cardiometabolic-pathways-linking-genetically-proxied-educational-attainment-to.md) - [Multimodal extraction of heart disease risk factors from free-text and structured records using Li](https://medichelpline.com/clinical-feed/plos-one-0-leveraging-free-text-clinical-records-for-heart-disease-classification-through.md) ## Navigation - [← Back to Cardiology Feed](https://medichelpline.com/clinical-feed/cardiology.md) - [← All Clinical Specialties](https://medichelpline.com/clinical-feed.md) ## Medical & Regulatory Disclaimer > [!CAUTION] > MedicHelpline content is structured for research, educational, and professional discovery purposes. It does not constitute individual medical advice, clinical diagnosis, or treatment recommendations. > Always verify dosing, contraindications, and regulatory alerts against official product labeling and primary regulatory sources before clinical decision-making.