Note2Chat: Improving LLMs for Multi-Turn Clinical History Taking Using Medical Notes

TL;DR

Note2Chat tackles the challenge of dynamic, multi-turn clinical history taking by leveraging widely available medical notes as supervision signals. It introduces a note-to-dialogue generation pipeline, a three-stage fine-tuning strategy (supervised fine-tuning, self-augmentation, and direct preference optimization), and a single-turn reasoning paradigm that enhances interpretability and sample efficiency. Experimental results show substantial gains in information recall and diagnostic accuracy over strong baselines such as GPT-4o, with Note2Chat-ST delivering efficient history gathering and Note2Chat-MT achieving higher recall. The work provides a scalable, note-driven approach for training LLMs in proactive history taking and differential diagnosis, with public release of code and dataset to support reproducibility and broader adoption.

Abstract

Effective clinical history taking is a foundational yet underexplored component of clinical reasoning. While large language models (LLMs) have shown promise on static benchmarks, they often fall short in dynamic, multi-turn diagnostic settings that require iterative questioning and hypothesis refinement. To address this gap, we propose \method{}, a note-driven framework that trains LLMs to conduct structured history taking and diagnosis by learning from widely available medical notes. Instead of relying on scarce and sensitive dialogue data, we convert real-world medical notes into high-quality doctor-patient dialogues using a decision tree-guided generation and refinement pipeline. We then propose a three-stage fine-tuning strategy combining supervised learning, simulated data augmentation, and preference learning. Furthermore, we propose a novel single-turn reasoning paradigm that reframes history taking as a sequence of single-turn reasoning problems. This design enhances interpretability and enables local supervision, dynamic adaptation, and greater sample efficiency. Experimental results show that our method substantially improves clinical reasoning, achieving gains of +16.9 F1 and +21.0 Top-1 diagnostic accuracy over GPT-4o. Our code and dataset can be found at https://github.com/zhentingsheng/Note2Chat.

Figures (3)

• Figure 1: Multi-turn clinical history taking is the dynamic process of gathering information from a patient; structured medical notes are the organized product, synthesizing that narrative into a concise, standardized record.
• Figure 2: Recall across symptom categories.
• Figure 3: Comparison of history taking performance between model and clinician.