Aloe: A Family of Fine-tuned Open Healthcare LLMs

TL;DR

Aloe introduces a family of open healthcare LLMs tuned for safety and performance at 7B–8B scale by combining model merging, two-stage alignment with Direct Preference Optimization (DPO), and advanced prompting. The approach leverages synthetic CoT data, extensive data cleaning, templating, and red-teaming to enhance factuality, ethics, and robustness, achieving state-of-the-art results for its size on several medical benchmarks. Extensive evaluation includes prompt engineering efficacy, medical-task performance, AI-principles metrics, and a comprehensive risk assessment, with the most capable variant released under CC-BY-NC 4.0 to support open research and reproducibility. The work highlights the practical trade-offs between ensembles, embedding choices, and computational costs, offering actionable guidance for developing responsible open healthcare LLMs with strong safety guarantees. Overall, Aloe demonstrates that strong performance at small scales is compatible with rigorous ethical considerations and transparent deployment in research contexts, advancing open healthcare AI while acknowledging residual risks and areas for improvement.

Abstract

As the capabilities of Large Language Models (LLMs) in healthcare and medicine continue to advance, there is a growing need for competitive open-source models that can safeguard public interest. With the increasing availability of highly competitive open base models, the impact of continued pre-training is increasingly uncertain. In this work, we explore the role of instruct tuning, model merging, alignment, red teaming and advanced inference schemes, as means to improve current open models. To that end, we introduce the Aloe family, a set of open medical LLMs highly competitive within its scale range. Aloe models are trained on the current best base models (Mistral, LLaMA 3), using a new custom dataset which combines public data sources improved with synthetic Chain of Thought (CoT). Aloe models undergo an alignment phase, becoming one of the first few policy-aligned open healthcare LLM using Direct Preference Optimization, setting a new standard for ethical performance in healthcare LLMs. Model evaluation expands to include various bias and toxicity datasets, a dedicated red teaming effort, and a much-needed risk assessment for healthcare LLMs. Finally, to explore the limits of current LLMs in inference, we study several advanced prompt engineering strategies to boost performance across benchmarks, yielding state-of-the-art results for open healthcare 7B LLMs, unprecedented at this scale.

Figures (8)

• Figure 1: Summary of Aloe training stages and data sources.
• Figure 2: Pipeline of the data processing for finetuning.
• Figure 3: Overview of the Medprompting scheme used for inference.
• Figure 4: ASR on (a) Aloe after the first DPO stage and (b) Llama3-Aloe-8B-Alpha , categorized per topic and style. Bold and italics on each figure mark total average ASR. Higher ASR means higher ratio of unsafe responses.
• Figure 5: DEITA scores for medical and general data.