Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia

Elisa Castagnari, Lillian Muyama, Adrien Coulet

TL;DR

Inspired by clinical guidelines, this study aimed to develop pathways similar to those that can be obtained in clinical guidelines, and tested three Large Language Models — Generative Pre-trained Transformer 4, Large Language Model Meta AI, andLLaMA — on a synthetic yet realistic dataset of 1000 patients to differentially diagnose anemia and its subtypes.

Abstract

In practice, clinicians achieve a diagnosis by following a sequence of steps, such as laboratory exams, observations, or imaging. The pathways to reach diagnosis decisions are documented by guidelines authored by expert organizations, which guide clinicians to reach a correct diagnosis through these sequences of steps. While these guidelines are beneficial for following medical reasoning and consolidating medical knowledge, they have some drawbacks. They often fail to address patients with uncommon conditions due to their focus on the majority population, and are slow and costly to update, making them unsuitable for rapidly emerging diseases or new practices. Inspired by clinical guidelines, our study aimed to develop pathways similar to those that can be obtained in clinical guidelines. We tested three Large Language Models (LLMs) -Generative Pretrained Transformer 4 (GPT-4), Large Language Model Meta AI (LLaMA), and Mistral -on a synthetic yet realistic dataset to differentially diagnose anemia and its subtypes. By using advanced prompting techniques to enhance the decision-making process, we generated diagnostic pathways using these models. Experimental results indicate that LLMs hold huge potential in clinical pathway discovery from patient data, with GPT-4 exhibiting the best performance in all conducted experiments.

Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia

TL;DR

Inspired by clinical guidelines, this study aimed to develop pathways similar to those that can be obtained in clinical guidelines, and tested three Large Language Models — Generative Pre-trained Transformer 4, Large Language Model Meta AI, andLLaMA — on a synthetic yet realistic dataset of 1000 patients to differentially diagnose anemia and its subtypes.

Abstract

In practice, clinicians achieve a diagnosis by following a sequence of steps, such as laboratory exams, observations, or imaging. The pathways to reach diagnosis decisions are documented by guidelines authored by expert organizations, which guide clinicians to reach a correct diagnosis through these sequences of steps. While these guidelines are beneficial for following medical reasoning and consolidating medical knowledge, they have some drawbacks. They often fail to address patients with uncommon conditions due to their focus on the majority population, and are slow and costly to update, making them unsuitable for rapidly emerging diseases or new practices. Inspired by clinical guidelines, our study aimed to develop pathways similar to those that can be obtained in clinical guidelines. We tested three Large Language Models (LLMs) -Generative Pretrained Transformer 4 (GPT-4), Large Language Model Meta AI (LLaMA), and Mistral -on a synthetic yet realistic dataset to differentially diagnose anemia and its subtypes. By using advanced prompting techniques to enhance the decision-making process, we generated diagnostic pathways using these models. Experimental results indicate that LLMs hold huge potential in clinical pathway discovery from patient data, with GPT-4 exhibiting the best performance in all conducted experiments.
Paper Structure (23 sections, 6 figures, 6 tables)

This paper contains 23 sections, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methods
  4. The Models
  5. Generative Pre-trained Transformer
  6. Large Language Model Meta AI (LLaMA)
  7. Mistral
  8. Prompting
  9. Plain Prompt
  10. Sequential Prompt
  11. Dataset
  12. Evaluation Approach and Implementation
  13. Results
  14. Plain Prompts
  15. Sequential Prompts
  16. ...and 8 more sections

Figures (6)

  • Figure 1: Anemia Decision Tree
  • Figure 2: Distribution of patients across anemia classes.
  • Figure 3: Pathways to Anemia of Chronic Disease (ACD) and Aplastic anemia, in pink and blue respectively, as generated by GPT-4.
  • Figure 4: The commonest pathway to each anemia class for GPT-4.
  • Figure 5: The commonest pathway to each anemia class for LLaMA.
  • ...and 1 more figures