The Multiple Ticket Hypothesis: Random Sparse Subnetworks Suffice for RLVR

TL;DR

The paper investigates whether random sparse subnetworks can match full RLVR finetuning by training only a tiny fraction of parameters (as little as $1\%$, i.e., $99\%$ sparsity). It introduces the Multiple Ticket Hypothesis, showing many independent random masks succeed with minimal overlap, and explains this via KL-constrained policy updates that effectively operate in a low-dimensional subspace defined by the Fisher information. Empirically, 20 random masks across multiple models and tasks achieve performance on par with dense finetuning, with a sharp degradation only when trainable parameters fall below about $0.01\%$. Theoretical analysis ties these results to a low-rank update geometry under KL constraints and delocalized eigenvectors, suggesting pretrained models harbor combinatorially many viable sparse tickets. Practically, this yields substantial efficiency gains and highlights fundamental redundancy in pretrained representations under RLVR objectives.

Abstract

The Lottery Ticket Hypothesis demonstrated that sparse subnetworks can match full-model performance, suggesting parameter redundancy. Meanwhile, in Reinforcement Learning with Verifiable Rewards (RLVR), recent work has shown that updates concentrate on a sparse subset of parameters, which further lends evidence to this underlying redundancy. We study the simplest possible way to exploit this redundancy: training only a randomly selected subset of parameters at extreme sparsities. Empirically, we find that training just 1\% of parameters matches or exceeds full-parameter RLVR finetuning across 3 models and 2 task domains. Moreover, different random masks show minimal overlap ($\leq 0.005$ Jaccard similarity) and yet all succeed, suggesting pretrained models contain many viable sparse subnetworks rather than one privileged set. We term this the Multiple Ticket Hypothesis. We explain this phenomenon through the implicit per-step KL constraint in RLVR, which restricts updates to a low-dimensional subspace, enabling arbitrary sparse masks to succeed.

Figures (7)

• Figure 1: Multiple random parameter subsets match or exceed full finetuning at 99% sparsity on Qwen-2.5-1.5B. Performance of 20 random parameter subsets of Qwen-2.5-1.5B across 100 training steps for GSM8K and MATH-500. 0% sparsity means full parameter finetuning. 99% sparsity indicates that 1% of parameters were trained.
• Figure 2: Multiple random parameter subsets match or exceed full finetuning at 99% sparsity on Qwen-2.5-0.5B. Performance of 20 random parameter subsets of Qwen-2.5-0.5B across 100 training steps for GSM8K and 150 steps for Alphabet sort.
• Figure 3: Random sparse training matches full finetuning at different sparsities. Validation performance across sparsity levels for three tasks. Error bars show variation across five random masks. Horizontal dashed lines indicate full-parameter baselines. All results use best learning rate from sweep for each configuration.
• Figure 4: Eigenspectrum analysis of the gradients.
• Figure 5: Comparison of random sparse training, full parameter finetuning and structured sparsity training on Qwen-2.5-1.5B.

Theorems & Definitions (9)

• Proposition 5.1: Low-Dimensional Policy Sensitivity
• Proposition 5.2: Sufficiency of Random Masks
• Remark 5.3
• Proposition 6.4: Restated
• proof
• Proposition 6.5: Restated
• proof
• Lemma 6.6: Concentration of Restricted Gram Matrix
• proof : Proof Sketch