Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Patient-Centric Communication: Leveraging LLMs to Simulate Patient Perspectives

Xinyao Ma, Rui Zhu, Zihao Wang, Jingwei Xiong, Qingyu Chen, Haixu Tang, L. Jean Camp, Lucila Ohno-Machado

TL;DR

This study investigates whether Large Language Models can simulate patient perspectives to improve patient understanding of discharge summaries. It employs in-context impersonation prompts to generate persona-specific responses and compares them to real human responses from a discharge-summary evaluation. Results show moderate overall alignment, with strong dependence on education level and text complexity, and highlight misalignment in perception-based tasks and longer documents. The findings suggest LLM-driven patient simulations can support scalable, patient-tailored communication, but biases and limitations must be addressed before clinical deployment. Future work includes enhancing accuracy, reducing bias, and integrating real-time feedback into healthcare workflows.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in role-playing scenarios, particularly in simulating domain-specific experts using tailored prompts. This ability enables LLMs to adopt the persona of individuals with specific backgrounds, offering a cost-effective and efficient alternative to traditional, resource-intensive user studies. By mimicking human behavior, LLMs can anticipate responses based on concrete demographic or professional profiles. In this paper, we evaluate the effectiveness of LLMs in simulating individuals with diverse backgrounds and analyze the consistency of these simulated behaviors compared to real-world outcomes. In particular, we explore the potential of LLMs to interpret and respond to discharge summaries provided to patients leaving the Intensive Care Unit (ICU). We evaluate and compare with human responses the comprehensibility of discharge summaries among individuals with varying educational backgrounds, using this analysis to assess the strengths and limitations of LLM-driven simulations. Notably, when LLMs are primed with educational background information, they deliver accurate and actionable medical guidance 88% of the time. However, when other information is provided, performance significantly drops, falling below random chance levels. This preliminary study shows the potential benefits and pitfalls of automatically generating patient-specific health information from diverse populations. While LLMs show promise in simulating health personas, our results highlight critical gaps that must be addressed before they can be reliably used in clinical settings. Our findings suggest that a straightforward query-response model could outperform a more tailored approach in delivering health information. This is a crucial first step in understanding how LLMs can be optimized for personalized health communication while maintaining accuracy.

Enhancing Patient-Centric Communication: Leveraging LLMs to Simulate Patient Perspectives

TL;DR

This study investigates whether Large Language Models can simulate patient perspectives to improve patient understanding of discharge summaries. It employs in-context impersonation prompts to generate persona-specific responses and compares them to real human responses from a discharge-summary evaluation. Results show moderate overall alignment, with strong dependence on education level and text complexity, and highlight misalignment in perception-based tasks and longer documents. The findings suggest LLM-driven patient simulations can support scalable, patient-tailored communication, but biases and limitations must be addressed before clinical deployment. Future work includes enhancing accuracy, reducing bias, and integrating real-time feedback into healthcare workflows.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in role-playing scenarios, particularly in simulating domain-specific experts using tailored prompts. This ability enables LLMs to adopt the persona of individuals with specific backgrounds, offering a cost-effective and efficient alternative to traditional, resource-intensive user studies. By mimicking human behavior, LLMs can anticipate responses based on concrete demographic or professional profiles. In this paper, we evaluate the effectiveness of LLMs in simulating individuals with diverse backgrounds and analyze the consistency of these simulated behaviors compared to real-world outcomes. In particular, we explore the potential of LLMs to interpret and respond to discharge summaries provided to patients leaving the Intensive Care Unit (ICU). We evaluate and compare with human responses the comprehensibility of discharge summaries among individuals with varying educational backgrounds, using this analysis to assess the strengths and limitations of LLM-driven simulations. Notably, when LLMs are primed with educational background information, they deliver accurate and actionable medical guidance 88% of the time. However, when other information is provided, performance significantly drops, falling below random chance levels. This preliminary study shows the potential benefits and pitfalls of automatically generating patient-specific health information from diverse populations. While LLMs show promise in simulating health personas, our results highlight critical gaps that must be addressed before they can be reliably used in clinical settings. Our findings suggest that a straightforward query-response model could outperform a more tailored approach in delivering health information. This is a crucial first step in understanding how LLMs can be optimized for personalized health communication while maintaining accuracy.
Paper Structure (22 sections, 1 equation, 5 figures, 2 tables)

This paper contains 22 sections, 1 equation, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Large Language Models
  4. Discharge Summary and LLM-based Summary Generation
  5. LLMs Personas
  6. Impersonation Methodology
  7. Prompting Large Language Model with Personas
  8. Human Subjects Discharge Summary Evaluation
  9. Results
  10. Data and Setup
  11. Effectiveness
  12. Impact of Categories in Discharge Summaries
  13. Misalignment Analysis
  14. Case Studies
  15. Broader Impact
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Four discharge summaries on different categories.
  • Figure 2: Effectiveness of the LLM's Ability to Simulate Personas Across Different Discharge Summary Categories
  • Figure 3: Results distribution for perception-based questions for DS1, DS2, DS3 and DS4. "A" to "E" represents {"A:Extremely easy", "B:Somewhat easy", "C:Neither easy nor difficult", "D:Somewhat difficult", "E:Extremely difficult"}, respectively.
  • Figure 4: Examples of Answers distributions of Selected Questions.
  • Figure 5: Examples of Answers distributions of Selected Questions.