Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Qibo: A Large Language Model for Traditional Chinese Medicine

Heyi Zhang, Xin Wang, Zhaopeng Meng, Zhe Chen, Pengwei Zhuang, Yongzhe Jia, Dawei Xu, Wenbin Guo

TL;DR

This work targets the gap in domain-specific LLMs for Traditional Chinese Medicine by developing Qibo, a TCM-focused LLM grounded in Chinese-LLaMA and trained through a two-stage pipeline (continuous pre-training followed by supervised instruction fine-tuning) on a 2GB TCM corpus. It introduces a comprehensive data processing scheme, constructs the Qibo-Benchmark for multi-dimensional evaluation, and builds a TCM consultation and syndrome differentiation workflow to improve interpretability and clinical relevance. Empirical results show Qibo achieves strong performance across subjective (win rates), objective (accuracy improvements), and NLP-task (Rouge-L) metrics relative to baselines, with notable gains at the 13B scale. The work provides a practical pathway for deploying interpretable, domain-specific LLMs in TCM and offers a valuable resource for evaluation in this specialized field.

Abstract

Large Language Models (LLMs) has made significant progress in a number of professional fields, including medicine, law, and finance. However, in traditional Chinese medicine (TCM), there are challenges such as the essential differences between theory and modern medicine, the lack of specialized corpus resources, and the fact that relying only on supervised fine-tuning may lead to overconfident predictions. To address these challenges, we propose a two-stage training approach that combines continuous pre-training and supervised fine-tuning. A notable contribution of our study is the processing of a 2GB corpus dedicated to TCM, constructing pre-training and instruction fine-tuning datasets for TCM, respectively. In addition, we have developed Qibo-Benchmark, a tool that evaluates the performance of LLM in the TCM on multiple dimensions, including subjective, objective, and three TCM NLP tasks. The medical LLM trained with our pipeline, named $\textbf{Qibo}$, exhibits significant performance boosts. Compared to the baselines, the average subjective win rate is 63%, the average objective accuracy improved by 23% to 58%, and the Rouge-L scores for the three TCM NLP tasks are 0.72, 0.61, and 0.55. Finally, we propose a pipline to apply Qibo to TCM consultation and demonstrate the model performance through the case study.

Qibo: A Large Language Model for Traditional Chinese Medicine

TL;DR

This work targets the gap in domain-specific LLMs for Traditional Chinese Medicine by developing Qibo, a TCM-focused LLM grounded in Chinese-LLaMA and trained through a two-stage pipeline (continuous pre-training followed by supervised instruction fine-tuning) on a 2GB TCM corpus. It introduces a comprehensive data processing scheme, constructs the Qibo-Benchmark for multi-dimensional evaluation, and builds a TCM consultation and syndrome differentiation workflow to improve interpretability and clinical relevance. Empirical results show Qibo achieves strong performance across subjective (win rates), objective (accuracy improvements), and NLP-task (Rouge-L) metrics relative to baselines, with notable gains at the 13B scale. The work provides a practical pathway for deploying interpretable, domain-specific LLMs in TCM and offers a valuable resource for evaluation in this specialized field.

Abstract

Large Language Models (LLMs) has made significant progress in a number of professional fields, including medicine, law, and finance. However, in traditional Chinese medicine (TCM), there are challenges such as the essential differences between theory and modern medicine, the lack of specialized corpus resources, and the fact that relying only on supervised fine-tuning may lead to overconfident predictions. To address these challenges, we propose a two-stage training approach that combines continuous pre-training and supervised fine-tuning. A notable contribution of our study is the processing of a 2GB corpus dedicated to TCM, constructing pre-training and instruction fine-tuning datasets for TCM, respectively. In addition, we have developed Qibo-Benchmark, a tool that evaluates the performance of LLM in the TCM on multiple dimensions, including subjective, objective, and three TCM NLP tasks. The medical LLM trained with our pipeline, named , exhibits significant performance boosts. Compared to the baselines, the average subjective win rate is 63%, the average objective accuracy improved by 23% to 58%, and the Rouge-L scores for the three TCM NLP tasks are 0.72, 0.61, and 0.55. Finally, we propose a pipline to apply Qibo to TCM consultation and demonstrate the model performance through the case study.
Paper Structure (27 sections, 2 equations, 4 figures, 8 tables)

This paper contains 27 sections, 2 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Large Language Models
  4. LLMs in Medical Domain
  5. Method
  6. Data Process
  7. Training Phase
  8. Continuous Pre-training
  9. Supervised Instruction Fine-Tuning
  10. TCM Consultation and Syndrome Differentiation
  11. Experiments and Evaluation
  12. Training Details
  13. Baselines
  14. Evaluation
  15. Subjective Evaluation
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Different diagnostic processes of TCM and modern medicine for the same sample.
  • Figure 2: The Overall Construction Process of Qibo.
  • Figure 3: The comparison results of Qibo and baseline models for the Safety, the Professionalism, and the Fluency, which is obtained through GPT-4.
  • Figure 4: The accuracy of different models in the 13 subject multiple-choice questions of the TCM Practicing Examination.