Masking Teacher and Reinforcing Student for Distilling Vision-Language Models

TL;DR

The paper tackles the challenge of distilling large vision-language models into compact, deployable ones by introducing Masters, a mask-progressive RL distillation framework. It combines (i) magnitude-based masking of the teacher with per-layer thresholds to shrink the active parameter space, (ii) a progressive restoration of teacher capacity to align with the student’s learning progress, and (iii) offline reinforcement learning with dual rewards (accuracy and distillation) using pre-generated responses to stabilize and accelerate knowledge transfer. The approach yields consistent gains over baselines across multiple benchmarks and model scales, and reveals that gradually increasing the teacher size during distillation can improve convergence and generalization. The framework emphasizes practicality for on-device intelligence by avoiding expensive think-answer cycles and maintaining fast inference, while offering a scalable path toward continually improving VLMs.

Abstract

Large-scale vision-language models (VLMs) have recently achieved remarkable multimodal understanding, but their massive size makes them impractical for deployment on mobile or edge devices. This raises the need for compact yet capable VLMs that can efficiently learn from powerful large teachers. However, distilling knowledge from a large teacher to a small student remains challenging due to their large size gap: the student often fails to reproduce the teacher's complex, high-dimensional representations, leading to unstable learning and degraded performance. To address this, we propose Masters (Masking Teacher and Reinforcing Student), a mask-progressive reinforcement learning (RL) distillation framework. Masters first masks non-dominant weights of the teacher to reduce unnecessary complexity, then progressively restores the teacher by gradually increasing its capacity during training. This strategy allows the student to learn richer representations from the teacher in a smooth and stable manner. To further refine knowledge transfer, Masters integrates an offline RL stage with two complementary rewards: an accuracy reward that measures the correctness of the generated responses, and a distillation reward that quantifies the ease of transferring responses from teacher to student. Unlike online think-answer RL paradigms that are computationally expensive and generate lengthy responses, our offline RL leverages pre-generated responses from masked teachers. These provide rich yet efficient guidance, enabling students to achieve strong performance without requiring the think-answer process.

Figures (6)

• Figure 1: Comparing Masters-applied VLMs with diverse open- and closed-source VLMs across broad model sizes for averaged performance (%) of numerous evaluation benchmarks: AI2D kembhavi2016diagram, ChartQA masry2022chartqa, MathVista lu2023mathvista, MMB liu2023mmbench, MM-Vet yu2023mm, MMMU yue2023mmmu, MMMU-Pro yue2024mmmu, MMStar chen2024we, BLINK fu2024blink, SEED-Bench li2023seed, SEED-Bench-2-Plus li2024seed, and RealWorldQA.
• Figure 2: Illustrating training dynamics of Masters, where we represent how (a) mask ratio is controlled during distillation, and (b) its averaged performance log of student (InternVL3.5-8B wang2025internvl3) for evaluation benchmarks in \ref{['tab:1']}. In addition, we show (c) the effect of RL under naive and mask-progressive distillation. Note that asterisk (*) represents the combined distillation of mid-size and large teacher.
• Figure 3: Overview of mask-progressive distillation where teacher is masked with a decreasing masking ratio (0.20, 0.15, 0.10, 0.05, 0), gradually restoring its full capacity. At each masking stage for teacher, student is updated using two rewards: accuracy reward $R_{\text{acc}}$ and distillation reward $R_{\text{distill}}$. This progressive distillation enables smooth and stable knowledge transfer to the student.
• Figure 4: Depicting multiple responses generated by both the masked teacher and the student, where an accuracy reward ($R_{\text{acc}}$) evaluates the binary correctness of each response, and a distillation reward ($R_{\text{distill}}$) measures the ease of knowledge transfer based on divergence objective between teacher and student logits. Note that the rank labels (1st, 2nd, 3rd, etc.) in $R_{\text{distill}}$ indicate the relative magnitude of the divergence values, where the smallest divergence (1st) receives the highest reward (1.0) and the largest divergence the lowest reward (0.0).
• Figure 5: Comparing the performances by the number of generated responses from teacher and student.