Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction

Mingyu Derek Ma, Xiaoxuan Wang, Yijia Xiao, Anthony Cuturrufo, Vijay S Nori, Eran Halperin, Wei Wang

TL;DR

MERA is introduced, a clinical diagnosis prediction model that bridges pertaining natural language clinical knowledge with medical practice and applies hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue.

Abstract

Clinical diagnosis prediction models, when provided with a patient's medical history, aim to detect potential diseases early, facilitating timely intervention and improving prognostic outcomes. However, the inherent scarcity of patient data and large disease candidate space often pose challenges in developing satisfactory models for this intricate task. The exploration of leveraging Large Language Models (LLMs) for encapsulating clinical decision processes has been limited. We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue. With concept memorization through fine-tuning, we bridge the natural language clinical knowledge with medical codes. Experimental results on MIMIC-III and IV datasets show that MERA achieves the state-of-the-art diagnosis prediction performance and dramatically elevates the diagnosis prediction capabilities of generative LMs.

Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction

TL;DR

MERA is introduced, a clinical diagnosis prediction model that bridges pertaining natural language clinical knowledge with medical practice and applies hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue.

Abstract

Clinical diagnosis prediction models, when provided with a patient's medical history, aim to detect potential diseases early, facilitating timely intervention and improving prognostic outcomes. However, the inherent scarcity of patient data and large disease candidate space often pose challenges in developing satisfactory models for this intricate task. The exploration of leveraging Large Language Models (LLMs) for encapsulating clinical decision processes has been limited. We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue. With concept memorization through fine-tuning, we bridge the natural language clinical knowledge with medical codes. Experimental results on MIMIC-III and IV datasets show that MERA achieves the state-of-the-art diagnosis prediction performance and dramatically elevates the diagnosis prediction capabilities of generative LMs.

Paper Structure

This paper contains 39 sections, 6 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Task Formulations
  4. Tasks.
  5. Input patient record.
  6. Medical code ontology as the decision space.
  7. Existing Paradigm of Generative LMs
  8. Mera: Learning to Memorize and Rank
  9. Medical Code Memorization
  10. Bidirectional code and definition memorization.
  11. Decision space structure memorization.
  12. Seq2seq Data Construction
  13. Diagnoses order perturbation.
  14. Learning Inter-visit Reasoning
  15. Hierarchical contrastive learning.
  16. ...and 24 more sections

Figures (1)

  • Figure 1: The model design of Mera. The diagnosis probability distribution is induced from token probabilities. It is optimized with hierarchical contrastive learning and dynamic cross-entropy losses.