Quantify Unmet Medical Need Across the Disease Landscape With LLM-Based Methodology

12 Mar 20264 min read0 viewsJournal Feed

GIST

by Elliott W. Sharp, Nicholas Fragola, Charlotte Blewitt, Matthew Goddeeris, Lee Lancashire, Charlie Hempstead, David C.

Fajgenbaum Background Despite the ultimate goal of medical researchers and funders being to maximize patient benefit, there is no systematic process for quantifying unmet medical need across diseases. While a relative unmet medical need scoring system would be valuable for prioritization of medical research, systematically performing this effort across all 22,701 human diseases is technically challenging, time-consuming, and expensive.

Using a large language model-based (LLM) architecture, we built a scalable method demonstrating feasibility to quantify “unmet medical need” criteria across all diseases, combine those criteria into a single weighted score, and extend the method into new criteria or diseases in the future. We aimed to quantitatively determine which diseases have the greatest unmet medical need and, therefore, which diseases are priority targets for new repurposed treatments.

Method and findings We defined 11 scoring criteria across three categories of unmet medical need.

Clinical Editorial

Purpose and scope

The study addresses the absence of a systematic approach to quantify unmet medical need across diseases.
It applies a large language model–based (LLM) architecture to assess unmet need criteria for all 22,701 human diseases cataloged in the MONDO ontology.
The objective is to identify priority targets for repurposed treatments by aggregating multiple criteria into a single weighted score.

Design and methodological approach

The authors define 11 scoring criteria distributed across three thematic categories of unmet medical need.
For each criterion, they experiment with various LLM models and refine prompts to generate disease-specific criterion scores.
A weighting scheme combines these criterion scores into an overall index per disease.
The development process used a 30-disease set to iterate prompting strategies and a separate 10-disease evaluation set held out to test the final prompt’s performance.
All 22,701 diseases were scored, enabling relative comparisons of unmet medical need across the entire disease space.

Performance, validity, and comparative context

Inter-expert agreement on scoring was strong, with 95% of ratings differing by no more than one point, indicating reliability of the framework.
Among tested LLMs, gpt-4o exhibited the closest alignment with expert rankings, demonstrated by smaller mean and dispersion of differences relative to human scores.
The study reports that LLM-derived scores correlated with expert assessments on key clinical dimensions, including mortality (Spearman ρ ≈ 0.845) and quality-adjusted life years lost (Spearman ρ ≈ 0.822), supporting the construct validity of the approach for prioritization purposes.
Data generation for the index proceeded expeditiously, taking roughly one hour with no missing data and a compute cost of about $120 USD.

Outcomes, accessibility, and practical relevance

The resulting Unmet Medical Need Index provides a quantitative basis for comparing diseases, enabling prioritization aligned with organizational aims.
The authors emphasize that the method is accessible and scalable, permitting customization through selection of criteria and their weights to suit different funders, institutions, or disease groups.
The index is positioned as a pragmatic, transparent tool to streamline research prioritization without relying on ad hoc judgments alone.

Limitations and cautions

A primary limitation is that the development and evaluation sets combined comprise only 40 diseases, constraining extrapolability and potentially affecting performance on less-represented disease areas.
The authors acknowledge that expanding the disease set would be valuable for future refinement and robustness of the scoring system.

Implications for future research and open questions

The approach can be extended by incorporating additional criteria or reweighting to reflect evolving priorities, funding landscapes, or clinical considerations.
Future work could explore applying the methodology to larger disease corpora or validating the index against real-world funding or treatment development trajectories.

Context and signaling for decision-makers

The study presents a reproducible, transparent framework that can be adopted by governments, universities, healthcare organizations, and disease groups to tailor prioritization of research efforts to unmet medical need.
While promising, decisions should account for the current limitation in disease sample size and consider iterative expansion and external validation as data availability grows.