This retrospective analysis used national Veterans Affairs (VA) data from 2017–2023 to evaluate 10 medication-based performance measures intended to assess outpatient cardiology clinician performance. Measures targeted common cardiovascular syndromes: coronary artery disease (CAD), heart failure with reduced ejection fraction (HFrEF), and atrial fibrillation (AF) with elevated stroke risk. The unit of analysis was the clinician–Veteran dyad at a cardiology encounter.
Investigators analyzed de-identified VA electronic health record data covering cardiology encounters from 2017 through 2023. The cohort included 456,071 unique cardiology encounters and 1,043 cardiology clinicians. Among encounters, 312,331 included CAD, 224,599 included HFrEF, and 178,678 included AF.
Achievement of each performance measure was defined as an active prescription of an indicated medication at the eligible patient encounter. Measure-level achievement was summarized at the clinician level across eligible encounters. The measures were medication-based, using prescription presence in the EHR as the indicator of guideline-directed therapy.
Measure reliability at the clinician level was estimated using the beta-binomial method. Reliability results were reported as values between 0 and 1, reflecting the proportion of observed variation attributable to between-clinician differences rather than random noise.
To evaluate whether patient case complexity confounded measure achievement, the authors used a Bayesian causal model to assess the impact of complexity on performance measure achievement. This model tested whether more complex cases were systematically associated with lower observed achievement on the medication-based measures.
A subset of cases (n = 952) underwent clinician peer review and served as the gold-standard comparator for three representative performance measures. Accuracy of the EHR-derived, medication-based measures was assessed against the peer review results, with balanced accuracy reported to summarize correct classification while accounting for class imbalance.
Median clinician-level measure achievement varied substantially across the 10 measures. The lowest reported median achievement was 0.23 for SGLT2 inhibitor prescribing in HFrEF, while the highest was 0.99 for beta blocker prescribing in HFrEF. Calculated reliability across measures ranged from 0.63 to 0.90, indicating that most measures demonstrated moderate to high reliability for distinguishing clinician-level performance.
The Bayesian causal model showed that case complexity did not significantly affect performance measure achievement. In other words, after modeling complexity, there was no meaningful evidence that higher patient complexity systematically reduced observed measure attainment at the clinician level in this VA cohort.
When compared with peer review (n = 952 cases) as the reference standard for three representative measures, the EHR-derived medication-based measures demonstrated poor to moderate accuracy. Balanced accuracy values ranged from 0.51 to 0.73. The measures tended to underestimate clinician performance relative to peer review, indicating misclassification bias in which the electronic-record-derived indicators missed instances that peer reviewers adjudicated as appropriate care.
The authors conclude that the 10 medication-based performance measures for CAD, HFrEF, and AF can be efficiently measured using electronic health record data and that most measures have sufficient reliability to support benchmarking and the identification of potential care gaps. However, because accuracy was suboptimal against clinician peer review, these measures should not be used alone as definitive assessments of individual clinician quality. They may be suitable for initial screening, quality improvement, or system-level benchmarking but warrant validation or supplementation (for example, targeted chart review) before high-stakes decisions about individual clinicians are made.
The study reports adherence to relevant ethical guidelines, that data were de-identified prior to analysis, and that necessary IRB/oversight approvals were obtained. Data can be shared following a written request to the authors and appropriate VA approvals in accordance with VA data security requirements. The authors declared no competing interests.