Primary care electronic health records (EHRs) often contain important clinical information embedded in free‑text notes that are not captured by structured coding systems. In Catalonia, routine primary care notes follow a semi‑structured format known as MEAP (in Catalan). The study addresses the extraction of six clinically relevant but uncoded urinary tract infection (UTI) features from MEAP narratives: fever, nitrites, leukocytes, lumbar pain, abdominal pain, and haematuria.
Extracting these features at scale is challenging because free text is variable and multilingual, and using cloud‑hosted commercial large language models raises privacy concerns and ongoing API costs. The authors therefore designed a local solution that aims to preserve confidentiality while providing automated structured data extraction for EHR research.
The extraction framework is a hybrid pipeline that combines an open‑weight compact small language model (SLM) with deterministic regular expressions applied after model inference. The chosen SLM was Phi4‑mini (3.8 billion parameters), deployed inside the institutional firewall using Ollama, so all computation occurred on secure local servers.
Prompts were iteratively refined in collaboration with clinicians to improve the model’s clinical extraction behavior. After the model produced outputs from MEAP notes, deterministic regular expressions were applied as a post‑processing step to normalize and finalize the identification of the six target features.
This hybrid approach was intended to balance the SLM’s ability to interpret variable narrative text with the precision and reproducibility of rule‑based pattern matching, while avoiding external data transfer and API expenditures.
The pipeline processed 15,498 MEAP narratives from a matched cohort of 2,962 primary care patients. From these narratives the system detected 4,663 occurrences of the six predefined clinical features.
The study compared feature occurrence between patients who progressed to acute pyelonephritis (cases) and matched controls who did not progress. Overall, cases exhibited a higher burden of the target features compared with controls, indicating the extracted uncoded features were clinically informative within the cohort analyzed.
Validation used two independent arms. First, a clinician gold standard consisted of a double‑blind review of 60 real patient records, with consensus resolution for disagreements. This provided a real‑world benchmark of pipeline performance against expert human review.
Second, an adversarial synthetic dataset of 720 notes was generated using GPT‑4.1 and Grok‑4.1 and enriched with linguistic noise to stress‑test the system under challenging narrative conditions. This synthetic arm evaluated robustness to noisy, diverse phrasing that may not be well represented in the real validation set.
Point estimates and 95% confidence intervals (CIs) were computed for standard classification metrics in both validation arms.
In the clinician‑validated real‑world arm, the pipeline achieved:
In the adversarial synthetic stress‑test, performance was:
Across both arms, the pipeline demonstrated very high specificity and PPV, while sensitivity was lower, particularly under synthetic adversarial noise and for narratively complex descriptions.
Within the processed dataset, patients who progressed to acute pyelonephritis had a greater prevalence of the extracted clinical features than non‑progressing controls (71.3% vs. 55.8%; standardized mean difference [SMD] = 0.326). The differences were most pronounced for fever (33.0% vs. 9.0%; SMD = 0.616) and lumbar pain (29.0% vs. 9.8%; SMD = 0.502). These results indicate the extracted uncoded features can reflect meaningful clinical differences relevant to disease progression.
All computation was performed within institutional servers; the model and pipeline ran behind the institutional firewall to avoid transferring patient data to external services. This design choice addresses privacy and regulatory concerns and also removes ongoing API costs associated with commercial cloud models.
Ethical oversight was obtained: the study protocol received approval from the Ethics Committee of IDIAP Jordi Gol (ethical approval code 22/089‑P; protocol IJG‑ITUCAT‑2022; approval date reported as 27 July 2022). The authors state that data were stored, processed, and analyzed in secure institutional servers and that appropriate patient consent and governance procedures were followed. The data produced are not publicly available due to institutional and regulatory restrictions on sharing patient‑level clinical data.
The authors conclude that a privacy‑preserving local hybrid framework combining a compact open‑weight SLM and regular expression rules can extract six uncoded UTI clinical features from multilingual primary care narratives with high precision and specificity while operating entirely on institutional infrastructure. The approach avoids API costs and external data exposure, making it practical for EHR research where privacy is paramount.
However, the pipeline showed reduced sensitivity for complex narrative descriptions and when challenged with adversarial linguistic noise in the synthetic dataset. The study reports published performance metrics and cohort outcomes but does not provide public access to patient‑level data or code; details on certain implementation parameters beyond those reported were not provided in the source.