Beyond Reasoning Gains: Mitigating General Capabilities Forgetting in Large Reasoning Models

TL;DR

The paper tackles general-capability forgetting in large reasoning models trained with RLVR by introducing RECAP, a replay-based strategy that also dynamically reweights objectives. RECAP replays general-capability data while online adjusting objective priorities using short-horizon convergence and instability signals, prioritizing underperforming or volatile tasks and down-weighting saturated ones. Across Qwen2.5-VL-3B and -7B, RECAP preserves baseline general abilities and often improves reasoning performance, outperforming common baselines and enabling more flexible trade-offs among rewards. The approach is end-to-end, plug-in, and requires no extra models or heavy tuning, with practical benefits such as shorter, more concise rationales and improved efficiency. Overall, RECAP offers a scalable, data-efficient means to maintain broad competencies during reasoning-centric post-training.

Abstract

Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning and has become a standard post-training paradigm for contemporary language and vision-language models. However, the RLVR recipe introduces a significant risk of capability regression, where models forget foundational skills after prolonged training without employing regularization strategies. We empirically confirm this concern, observing that open-source reasoning models suffer performance degradation on core capabilities such as perception and faithfulness. While imposing regularization terms like KL divergence can help prevent deviation from the base model, these terms are calculated on the current task, thus they do not guarantee broader knowledge. Meanwhile, commonly used experience replay across heterogeneous domains makes it nontrivial to decide how much training focus each objective should receive. To address this, we propose RECAP-a replay strategy with dynamic objective reweighting for general knowledge preservation. Our reweighting mechanism adapts in an online manner using short-horizon signals of convergence and instability, shifting the post-training focus away from saturated objectives and toward underperforming or volatile ones. Our method is end-to-end and readily applicable to existing RLVR pipelines without training additional models or heavy tuning. Extensive experiments on benchmarks based on Qwen2.5-VL-3B and Qwen2.5-VL-7B demonstrate the effectiveness of our method, which not only preserves general capabilities but also improves reasoning by enabling more flexible trade-offs among in-task rewards.

Figures (11)

• Figure 1: General capabilities comparison of base VLMs (blue) and their reasoning-tuned variants (green/purple/red) on six representative, non-reasoning benchmarks (higher is better): A-OKVQA (knowledge-based VQA), AesBench Huang2024AesBench (image aesthetics) , VStar Wu2023VStar (spatio-temporal reasoning), VisOnly Kamoi2025VisOnlyQA (vision-only recognition aggregate) , OCRBench liu2024ocrbench (text recognition), and R-Bench-Dis li2025r (distribution-shift robustness). Across both Qwen2.5-VL families, reasoning-finetuned models generally underperform their base models on perception and robustness tasks, whereas MiMo-VL-7B-RL remains close to its SFT baseline.
• Figure 2: Performance comparison between the model finetuned solely on math data and with RECAP. Our model not only preserves the base model performance but also improves it (2%) while the reasoning model quickly falls behind the base model after 100 iterations.
• Figure 3: Overview of our RECAP. Along with the target reasoning task, we sample data from general domains in order to maintain that knowledge during finetuning. Initially, the objectives of interest are reweighted uniformly to optimize the main model. After a few iterations, we record the convergence behavior of individual objectives. Based on this, we adjust the focus to prevent the dominance of any objectives and put less weight on saturated ones.
• Figure 4: Different rewards have different behaviors of convergence. While the format reward is easy to optimize and obtains the highest rate of convergence, it quickly saturates and thus yields low convergence rate ($c\sim1$) and instability ($i\sim0$) after 50 steps. By contrast, the reasoning accuracy fluctuates the most, thereby steering the optimization toward the corresponding objective. IoU and ntp indicate the IoU reward and the next-token-prediction accuracy during training. The result is obtained on the first setting in our experiment section below.
• Figure 5: Evolution of format and accuracy rewards on the reasoning domain during training: Training curves for the format reward over full training (left), an early-training zoom (middle), and the accuracy reward (right). While the uniform baseline is better in maximizing the format reward, it falls behind our proposed method later in terms of accuracy, as we prioritize correct solutions over formatting once the model can follow the predefined template. Curves are smoothed with an exponential moving average for readability.