Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models

TL;DR

This work tackles the challenge of building open-source medical language capabilities by introducing Aquila-Med, a bilingual medical LLM built atop Aquila through a full pipeline of continue pre-training, supervised fine-tuning, and reinforcement learning from human feedback. It contributes large-scale bilingual medical data for pre-training, a high-quality SFT set, and Direct Preference Optimization–based alignment, all open-sourced alongside the training workflow. Evaluations across medical knowledge benchmarks and multi-turn medical dialogues show improved medical grounding and conversational quality compared with baselines. The release of datasets and training processes aims to accelerate responsible, open innovation in medical AI research and development.

Abstract

Recently, both closed-source LLMs and open-source communities have made significant strides, outperforming humans in various general domains. However, their performance in specific professional fields such as medicine, especially within the open-source community, remains suboptimal due to the complexity of medical knowledge. We propose Aquila-Med, a bilingual medical LLM based on Aquila, addressing these challenges through continue pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF). We construct a large-scale Chinese and English medical dataset for continue pre-training and a high-quality SFT dataset, covering extensive medical specialties. Additionally, we develop a high-quality Direct Preference Optimization (DPO) dataset for further alignment. Aquila-Med achieves notable results across single-turn, multi-turn dialogues, and medical multiple-choice questions, demonstrating the effectiveness of our approach. We open-source the datasets and the entire training process, contributing valuable resources to the research community. Our models and datasets will released at https://huggingface.co/BAAI/AquilaMed-RL.

Figures (6)

• Figure 1: The overall pipline of Aquila-Med-Chat (RL), which includes the continue pre-training, supervised fine-tuning, and the DPO process.
• Figure 2: Statistics on the distribution of our proposed SFT dataset.
• Figure 3: The comparison of our model's predicted answers and the ground truth answers from the dataset on single-round dialogues from the Huatuo MedicalQA.
• Figure 4: Performance on the CMTMedQA dataset in multi-round dialogues. The x-axis represents different rounds of the dialogue, while the "Avg" data point displays the average score across all rounds.
• Figure 5: Performance on the CMT-Clin dataset in multi-round dialogues. The x-axis represents different rounds of the dialogue, while the "Avg" data point displays the average score across all rounds.