Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

Yue Jiang, Jiawei Chen, Dingkang Yang, Mingcheng Li, Shunli Wang, Tong Wu, Ke Li, Lihua Zhang

TL;DR

CoMT introduces a chain-of-medical-thought framework that mimics radiologists' diagnostic reasoning by decomposing reports into hierarchical medical clues and constructing chain-based QA pairs to guide MRG. This cognitive-structure approach reduces hallucinations and improves both objective and human-evaluated quality on OpenI and MIMIC-CXR, with demonstrated out-of-distribution benefits. Two new CoMT-based datasets are released to support further research. Overall, CoMT enhances the clinical reliability of LVLM-driven medical report generation by aligning generation with structured medical reasoning.

Abstract

Automatic medical report generation (MRG), which possesses significant research value as it can aid radiologists in clinical diagnosis and report composition, has garnered increasing attention. Despite recent progress, generating accurate reports remains arduous due to the requirement for precise clinical comprehension and disease diagnosis inference. Furthermore, owing to the limited accessibility of medical data and the imbalanced distribution of diseases, the underrepresentation of rare diseases in training data makes large-scale medical visual language models (LVLMs) prone to hallucinations, such as omissions or fabrications, severely undermining diagnostic performance and further intensifying the challenges for MRG in practice. In this study, to effectively mitigate hallucinations in medical report generation, we propose a chain-of-medical-thought approach (CoMT), which intends to imitate the cognitive process of human doctors by decomposing diagnostic procedures. The radiological features with different importance are structured into fine-grained medical thought chains to enhance the inferential ability during diagnosis, thereby alleviating hallucination problems and enhancing the diagnostic accuracy of MRG. The code and dataset have been released at https://github.com/FRENKIE-CHIANG/CoMT.

CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

TL;DR

CoMT introduces a chain-of-medical-thought framework that mimics radiologists' diagnostic reasoning by decomposing reports into hierarchical medical clues and constructing chain-based QA pairs to guide MRG. This cognitive-structure approach reduces hallucinations and improves both objective and human-evaluated quality on OpenI and MIMIC-CXR, with demonstrated out-of-distribution benefits. Two new CoMT-based datasets are released to support further research. Overall, CoMT enhances the clinical reliability of LVLM-driven medical report generation by aligning generation with structured medical reasoning.

Abstract

Automatic medical report generation (MRG), which possesses significant research value as it can aid radiologists in clinical diagnosis and report composition, has garnered increasing attention. Despite recent progress, generating accurate reports remains arduous due to the requirement for precise clinical comprehension and disease diagnosis inference. Furthermore, owing to the limited accessibility of medical data and the imbalanced distribution of diseases, the underrepresentation of rare diseases in training data makes large-scale medical visual language models (LVLMs) prone to hallucinations, such as omissions or fabrications, severely undermining diagnostic performance and further intensifying the challenges for MRG in practice. In this study, to effectively mitigate hallucinations in medical report generation, we propose a chain-of-medical-thought approach (CoMT), which intends to imitate the cognitive process of human doctors by decomposing diagnostic procedures. The radiological features with different importance are structured into fine-grained medical thought chains to enhance the inferential ability during diagnosis, thereby alleviating hallucination problems and enhancing the diagnostic accuracy of MRG. The code and dataset have been released at https://github.com/FRENKIE-CHIANG/CoMT.
Paper Structure (14 sections, 3 equations, 2 figures, 4 tables)

This paper contains 14 sections, 3 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Overview of the CoMT
  5. Inspiration for the CoMT
  6. Application Details of CoMT
  7. Evaluation Metrics
  8. Experiment and Discussion
  9. New Datasets Construction
  10. Effectiveness and Superiority of CoMT
  11. Generalizability of CoMT on Out-of-Distribution Dataset
  12. Effects of CoMT on Different Types of Hallucinations
  13. Conclusions
  14. Acknowledgements

Figures (2)

  • Figure 1: Illustration of CoMT's process for constructing hierarchical QA pairs based on real clinical image reports. (a). The progressive cognitive process of doctors for a medical image. (b). End-to-end data reconstruction of medical reports. (c). Construction of Hierarchical QA pairs. (d). Chain-based QA Pair Refactoring.
  • Figure 2: The percentage change in the number of different types of hallucinations after using the CoMT method. "w/o" stands for without.