IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment

TL;DR

This work introduces a medical instruction dataset, CMedINS, containing six medical instructions derived from actual medical tasks, which effectively fine-tunes LLM in conjunction with other data, and launches the medical model, IIMedGPT, employing an efficient preference alignment method, Direct preference Optimization(DPO).

Abstract

Recent researches of large language models(LLM), which is pre-trained on massive general-purpose corpora, have achieved breakthroughs in responding human queries. However, these methods face challenges including limited data insufficiency to support extensive pre-training and can not align responses with users' instructions. To address these issues, we introduce a medical instruction dataset, CMedINS, containing six medical instructions derived from actual medical tasks, which effectively fine-tunes LLM in conjunction with other data. Subsequently, We launch our medical model, IIMedGPT, employing an efficient preference alignment method, Direct preference Optimization(DPO). The results show that our final model outperforms existing medical models in medical dialogue.Datsets, Code and model checkpoints will be released upon acceptance.

Figures (7)

• Figure 1: Overall structure of our proposed pipeline.
• Figure 2: The distribution of CMedINS dataset
• Figure 3: Example of the instruction pair. The query part is from real medical records.
• Figure 4: Experiments of our model on the evaluation dataset. Left column indicates result that our model after SFT. Right column indicates result that our model after SFT and DPO.
• Figure 5: Ablation Experiment of IIMedGPT.w. represents the model winning after the DPO process.w/o represents the model winning before the DPO process.