When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models

TL;DR

This study investigates how the length of supervised fine-tuning data affects short-context reasoning in long-context LLMs. By decoupling MHA and FFN modules and applying a knowledge-conflict framework, the authors show that long-context SFT can improve short-context performance while independently strengthening both modules, yet it biases knowledge toward contextual cues. Hybrid training that mixes long- and short-context data mitigates these biases and yields robust, task-dependent gains. The findings challenge the view that long-context SFT harms short-context capabilities and offer actionable guidance for designing SFT regimens in long-context models.

Abstract

Large language models (LLMs) have achieved impressive performance across natural language processing (NLP) tasks. As real-world applications increasingly demand longer context windows, continued pretraining and supervised fine-tuning (SFT) on long-context data has become a common approach. While the effects of data length in continued pretraining have been extensively studied, their implications for SFT remain unclear. In this work, we systematically investigate how SFT data length influences LLM behavior on short-context tasks. Counterintuitively, we find that long-context SFT improves short-context performance, contrary to the commonly observed degradation from long-context pretraining. To uncover the underlying mechanisms of this phenomenon, we first decouple and analyze two key components, Multi-Head Attention (MHA) and Feed-Forward Network (FFN), and show that both independently benefit from long-context SFT. We further study their interaction and reveal a knowledge preference bias: long-context SFT promotes contextual knowledge, while short-context SFT favors parametric knowledge, making exclusive reliance on long-context SFT suboptimal. Finally, we demonstrate that hybrid training mitigates this bias, offering explainable guidance for fine-tuning LLMs.

Figures (12)

• Figure 1: Performance comparison of MHA module replacement. Accuracy is measured on the GSM8K.
• Figure 2: Performance comparison of FFN module replacement. Accuracy is measured on the World Capital.
• Figure 3: Heatmap of ChatQA2-SFT model Retrieval score, UltraChat-SFT model Retrieval score and their retrieval score difference, computed as ChatQA2-SFT retrieval score minus UltraChat-SFT retrieval score.
• Figure 4: Entropy difference of each attention layer, computed as UltraChat-SFT entropy minus ChatQA2-SFT entropy. Green points indicate layers where ChatQA2-SFT shows higher confidence (lower answer entropy) while maintaining flexibility in reasoning (higher reasoning entropy).
• Figure 5: Relative Difference of Activation Mean, Variance, and Sparsity between two models. Computed as $\Delta = \frac{m_c - m_u}{m_u}$, where $m_c,m_u$ represents metrics from ChatQA2-SFT and UltraChat-SFT model respectively.