Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Hybrid Framework with Large Language Models for Rare Disease Phenotyping

Jinge Wu, Hang Dong, Zexi Li, Haowei Wang, Runci Li, Arijit Patra, Chengliang Dai, Waqar Ali, Phil Scordis, Honghan Wu

TL;DR

A novel hybrid framework that integrates the Orphanet Rare Disease Ontology (ORDO) and the Unified Medical Language System (UMLS) to create a comprehensive rare disease vocabulary is proposed and demonstrates superior performance and the potential to uncover hidden rare disease cases.

Abstract

Rare diseases pose significant challenges in diagnosis and treatment due to their low prevalence and heterogeneous clinical presentations. Unstructured clinical notes contain valuable information for identifying rare diseases, but manual curation is time-consuming and prone to subjectivity. This study aims to develop a hybrid approach combining dictionary-based natural language processing (NLP) tools with large language models (LLMs) to improve rare disease identification from unstructured clinical reports. We propose a novel hybrid framework that integrates the Orphanet Rare Disease Ontology (ORDO) and the Unified Medical Language System (UMLS) to create a comprehensive rare disease vocabulary. The proposed hybrid approach demonstrates superior performance compared to traditional NLP systems and standalone LLMs. Notably, the approach uncovers a significant number of potential rare disease cases not documented in structured diagnostic records, highlighting its ability to identify previously unrecognized patients.

A Hybrid Framework with Large Language Models for Rare Disease Phenotyping

TL;DR

A novel hybrid framework that integrates the Orphanet Rare Disease Ontology (ORDO) and the Unified Medical Language System (UMLS) to create a comprehensive rare disease vocabulary is proposed and demonstrates superior performance and the potential to uncover hidden rare disease cases.

Abstract

Rare diseases pose significant challenges in diagnosis and treatment due to their low prevalence and heterogeneous clinical presentations. Unstructured clinical notes contain valuable information for identifying rare diseases, but manual curation is time-consuming and prone to subjectivity. This study aims to develop a hybrid approach combining dictionary-based natural language processing (NLP) tools with large language models (LLMs) to improve rare disease identification from unstructured clinical reports. We propose a novel hybrid framework that integrates the Orphanet Rare Disease Ontology (ORDO) and the Unified Medical Language System (UMLS) to create a comprehensive rare disease vocabulary. The proposed hybrid approach demonstrates superior performance compared to traditional NLP systems and standalone LLMs. Notably, the approach uncovers a significant number of potential rare disease cases not documented in structured diagnostic records, highlighting its ability to identify previously unrecognized patients.
Paper Structure (25 sections, 1 equation, 5 figures, 2 tables)

This paper contains 25 sections, 1 equation, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Rare Disease Identification as Text Phenotyping with Ontologies
  4. Natural Language Processing for Text Phenotyping
  5. Methods
  6. Problem Statement
  7. Rare Disease Terminology from Ontology
  8. Dictionary-Based Text Phenotyping
  9. LLM-Based Text Phenotyping
  10. Experiments Design
  11. Dataset
  12. Data Annotation
  13. Experiment Setup
  14. Baseline
  15. BERT-based models:
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Overview of our work. In the given example report, there are some abbreviations that are mistakenly extracted as rare diseases (eg. PNA: Pneumonia, SAR: Subacute rehabilitation), and also some negated mentioned are extracted (eg. no signs of xxx). We propose leveraging LLMs for enhanced contextual filtering, enabling more precise determinations of relevance and validity within the extracted information.
  • Figure 2: Data Distribution of MIMIC-IV's discharge summary lengths.
  • Figure 3: Few-shot prompting performance (F1) of LLMs.
  • Figure 4: Impact of context length on the performance (F1) of LLMs.
  • Figure 5: Case identification of rare diseases identified by NLP-based (free-text) and ICD-based (structured) data.