Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Domain Pretraining Interferes with Instruction Alignment: An Empirical Study of Adapter Merging in Medical LLMs

Junyi Zou

TL;DR

This paper addresses the challenge of combining broad medical knowledge with safe instruction following in LLMs by proposing a two-stage LoRA pipeline (domain-adaptive pre-training followed by supervised fine-tuning) and a linear Weighted Adapter Merging strategy to fuse the resulting adapters. It reveals an adapter interference phenomenon where surface metrics like BLEU-4 decline as domain-knowledge weight increases, while medical reasoning performance (e.g., MedQA) can improve, highlighting misalignment between common benchmarks and real reasoning capabilities. The key contribution is the Weighted Adapter Merging formulation, $W_{\text{merged}} = W_{\text{base}} + \alpha_{\text{PT}} \Delta W_{\text{PT}} + \alpha_{\text{SFT}} \Delta W_{\text{SFT}}$ with $\alpha_{\text{SFT}}=0.7$ and $\alpha_{\text{PT}}=0.3$, enabling a controlled trade-off between knowledge injection and alignment. The findings underscore safety benefits and a latent 'Thinking' mode artifact in mixed models, providing practical guidance for deploying safety-critical medical LLMs under parameter-efficient fine-tuning regimes.

Abstract

Large language models (LLMs) show strong general capability but often struggle with medical terminology precision and safety-critical instruction following. We present a case study for adapter interference in safety-critical domains using a 14B-parameter base model through a two-stage LoRA pipeline: (1) domain-adaptive pre-training (PT) to inject broad medical knowledge via continued pre-training (DAPT), and (2) supervised fine-tuning (SFT) to align the model with medical question-answering behaviors through instruction-style data. To balance instruction-following ability and domain knowledge retention, we propose Weighted Adapter Merging, linearly combining SFT and PT adapters before exporting a merged base-model checkpoint. On a held-out medical validation set (F5/F6), the merged model achieves BLEU-4 = 16.38, ROUGE-1 = 20.42, ROUGE-2 = 4.60, and ROUGE-L = 11.54 under a practical decoding configuration. We further analyze decoding sensitivity and training stability with loss curves and controlled decoding comparisons.

When Domain Pretraining Interferes with Instruction Alignment: An Empirical Study of Adapter Merging in Medical LLMs

TL;DR

This paper addresses the challenge of combining broad medical knowledge with safe instruction following in LLMs by proposing a two-stage LoRA pipeline (domain-adaptive pre-training followed by supervised fine-tuning) and a linear Weighted Adapter Merging strategy to fuse the resulting adapters. It reveals an adapter interference phenomenon where surface metrics like BLEU-4 decline as domain-knowledge weight increases, while medical reasoning performance (e.g., MedQA) can improve, highlighting misalignment between common benchmarks and real reasoning capabilities. The key contribution is the Weighted Adapter Merging formulation, with and , enabling a controlled trade-off between knowledge injection and alignment. The findings underscore safety benefits and a latent 'Thinking' mode artifact in mixed models, providing practical guidance for deploying safety-critical medical LLMs under parameter-efficient fine-tuning regimes.

Abstract

Large language models (LLMs) show strong general capability but often struggle with medical terminology precision and safety-critical instruction following. We present a case study for adapter interference in safety-critical domains using a 14B-parameter base model through a two-stage LoRA pipeline: (1) domain-adaptive pre-training (PT) to inject broad medical knowledge via continued pre-training (DAPT), and (2) supervised fine-tuning (SFT) to align the model with medical question-answering behaviors through instruction-style data. To balance instruction-following ability and domain knowledge retention, we propose Weighted Adapter Merging, linearly combining SFT and PT adapters before exporting a merged base-model checkpoint. On a held-out medical validation set (F5/F6), the merged model achieves BLEU-4 = 16.38, ROUGE-1 = 20.42, ROUGE-2 = 4.60, and ROUGE-L = 11.54 under a practical decoding configuration. We further analyze decoding sensitivity and training stability with loss curves and controlled decoding comparisons.
Paper Structure (25 sections, 1 equation, 3 figures, 3 tables)

This paper contains 25 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Domain-Adaptive Pre-training (DAPT)
  4. Instruction Tuning and Alignment
  5. Parameter-Efficient Fine-Tuning (PEFT)
  6. Model Merging and Adapter Composition
  7. Methodology
  8. Data Preparation and Leakage Audit
  9. Base Model
  10. Stage 1: Domain-Adaptive Pre-training (PT)
  11. Objective and Intuition
  12. Data Shape and Training Constraints
  13. Stability and Monitoring
  14. Stage 2: Supervised Fine-Tuning (SFT)
  15. Objective and Loss Masking
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Loss curves for the Domain-Adaptive Pre-training (PT) stage.
  • Figure 2: Loss curves for the Supervised Fine-Tuning (SFT) stage.
  • Figure 3: The Interference Phenomenon. As PT weight increases, surface metrics (BLEU-4) collapse due to style mismatch (CoT artifacts), but reasoning accuracy (MedQA, CMExam) actually improves or remains high. This suggests that surface metrics are misaligned with capability in medical reasoning tasks.