Revisiting Catastrophic Forgetting in Large Language Model Tuning

TL;DR

This paper investigates catastrophic forgetting during instruction fine-tuning of large language models and reveals a direct link between forgetting and the loss landscape's flatness. By visualizing loss surfaces and computing flatness metrics, the authors show that sharper loss landscapes correlate with stronger CF and greater generalization drops, especially as task gaps widen. They propose Sharpness-Aware Minimization (SAM), solving min_w max_{||ε||_2≤ρ} f(w+ε) to flatten the landscape and mitigate CF, with a two-step update and ρ=2 in practice. Empirical results across multiple datasets and model sizes demonstrate SAM’s effectiveness in reducing CF, its robustness relative to size, and its complementary compatibility with existing anti-forgetting methods, supported by reproducibility resources and public code.

Abstract

Catastrophic Forgetting (CF) means models forgetting previously acquired knowledge when learning new data. It compromises the effectiveness of large language models (LLMs) during fine-tuning, yet the underlying causes have not been thoroughly investigated. This paper takes the first step to reveal the direct link between the flatness of the model loss landscape and the extent of CF in the field of LLMs. Based on this, we introduce the sharpness-aware minimization to mitigate CF by flattening the loss landscape. Experiments on three widely-used fine-tuning datasets, spanning different model scales, demonstrate the effectiveness of our method in alleviating CF. Analyses show that we nicely complement the existing anti-forgetting strategies, further enhancing the resistance of LLMs to CF.

Figures (1)

• Figure 1: Visualization of different models' loss landscapes (LLS) with contour lines. We can see that with the data/task gaps of (a), (b), (c) gradually increase, the disturbance of their contours becomes more obvious.