Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PatientHub: A Unified Framework for Patient Simulation

Sahand Sabour, TszYam NG, Minlie Huang

TL;DR

Fragmentation in patient-simulation methods currently impedes reproducibility and fair benchmarking. The paper introduces PatientHub, a unified modular framework with four core components—Clients and Therapists, Evaluators, Generators, and Events—plus a graph-based dialogue orchestrator and rubric-driven LLM-as-a-judge for cross-method evaluation. It provides standardized abstractions, logging, and open-source implementations to enable synthetic data generation, benchmarking, and rapid prototyping of new simulators, demonstrated across multiple CBT-oriented methods. The framework enables controlled comparisons across simulators and therapists, accelerates method development, and supports scalable synthetic data pipelines for patient-centered dialogue research. Overall, PatientHub offers a practical foundation for reproducible benchmarks, datasets, and future method development in mental-health dialogue systems.

Abstract

As Large Language Models increasingly power role-playing applications, simulating patients has become a valuable tool for training counselors and scaling therapeutic assessment. However, prior work is fragmented: existing approaches rely on incompatible, non-standardized data formats, prompts, and evaluation metrics, hindering reproducibility and fair comparison. In this paper, we introduce PatientHub, a unified and modular framework that standardizes the definition, composition, and deployment of simulated patients. To demonstrate PatientHub's utility, we implement several representative patient simulation methods as case studies, showcasing how our framework supports standardized cross-method evaluation and the seamless integration of custom evaluation metrics. We further demonstrate PatientHub's extensibility by prototyping two new simulator variants, highlighting how PatientHub accelerates method development by eliminating infrastructure overhead. By consolidating existing work into a single reproducible pipeline, PatientHub lowers the barrier to developing new simulation methods and facilitates cross-method and cross-model benchmarking. Our framework provides a practical foundation for future datasets, methods, and benchmarks in patient-centered dialogue, and the code is publicly available via https://github.com/Sahandfer/PatientHub.

PatientHub: A Unified Framework for Patient Simulation

TL;DR

Fragmentation in patient-simulation methods currently impedes reproducibility and fair benchmarking. The paper introduces PatientHub, a unified modular framework with four core components—Clients and Therapists, Evaluators, Generators, and Events—plus a graph-based dialogue orchestrator and rubric-driven LLM-as-a-judge for cross-method evaluation. It provides standardized abstractions, logging, and open-source implementations to enable synthetic data generation, benchmarking, and rapid prototyping of new simulators, demonstrated across multiple CBT-oriented methods. The framework enables controlled comparisons across simulators and therapists, accelerates method development, and supports scalable synthetic data pipelines for patient-centered dialogue research. Overall, PatientHub offers a practical foundation for reproducible benchmarks, datasets, and future method development in mental-health dialogue systems.

Abstract

As Large Language Models increasingly power role-playing applications, simulating patients has become a valuable tool for training counselors and scaling therapeutic assessment. However, prior work is fragmented: existing approaches rely on incompatible, non-standardized data formats, prompts, and evaluation metrics, hindering reproducibility and fair comparison. In this paper, we introduce PatientHub, a unified and modular framework that standardizes the definition, composition, and deployment of simulated patients. To demonstrate PatientHub's utility, we implement several representative patient simulation methods as case studies, showcasing how our framework supports standardized cross-method evaluation and the seamless integration of custom evaluation metrics. We further demonstrate PatientHub's extensibility by prototyping two new simulator variants, highlighting how PatientHub accelerates method development by eliminating infrastructure overhead. By consolidating existing work into a single reproducible pipeline, PatientHub lowers the barrier to developing new simulation methods and facilitates cross-method and cross-model benchmarking. Our framework provides a practical foundation for future datasets, methods, and benchmarks in patient-centered dialogue, and the code is publicly available via https://github.com/Sahandfer/PatientHub.
Paper Structure (25 sections, 1 figure, 4 tables)

This paper contains 25 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Patient Simulation Methods
  4. Applications of Patient Simulation
  5. PatientHub
  6. Core Components
  7. Clients and Therapists
  8. Evaluators
  9. Generators
  10. Events
  11. Technical Details
  12. Use Cases
  13. Synthetic Data Generation
  14. Benchmarking and Evaluation
  15. Developing New Methods
  16. ...and 10 more sections

Figures (1)

  • Figure 1: An Overview of PatientHub. Our framework includes (i) a unified collection of personas, prompts, and configurations from existing work; (ii) support for several agent roles for interactions; (iii) orchestration of dialogue flows/events as directed graphs; and (iv) cross-method benchmarking through a modular LLM-as-a-Judge Evaluator supporting four fundamental evaluation paradigms.