GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR

TL;DR

This work tackles RLVR optimization challenges by recognizing a geometry-aware, low-rank structure in RL updates. It introduces GeoRA, which constructs a geometry-constrained view $W_{Geo}$, initializes $A_{Geo}$ and $B_{Geo}$ from the top-$r$ singular directions, and enforces a frozen residual $W_{res}$ to preserve pre-trained knowledge, enabling dense computation and improved stability. Through SVD-based initialization and a masking-based prior, GeoRA achieves state-of-the-art performance among PEFT methods on mathematical benchmarks and demonstrates strong out-of-domain generalization and resistance to catastrophic forgetting on Qwen and Llama, while delivering substantial efficiency gains (roughly $99.5\%$ fewer trainable parameters, $28.5\%$ less VRAM, and $\sim20\%$ faster training). Mechanistic analyses confirm that RLVR updates are intrinsically low-rank and directional, and GeoRA’s geometry-aligned updates yield lower spectral distortion (NSS) and favorable subspace alignment, explaining stability and performance gains. Overall, GeoRA provides a principled, hardware-friendly pathway for robust RLVR fine-tuning that preserves pre-trained structure while enabling efficient adaptation.

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) is crucial for advancing large-scale reasoning models. However, existing parameter-efficient methods, such as PiSSA and MiLoRA, are designed for Supervised Fine-Tuning (SFT) and do not account for the distinct optimization dynamics and geometric structures of RLVR. Applying these methods directly leads to spectral collapse and optimization instability, which severely limit model performance. Meanwhile, alternative approaches that leverage update sparsity encounter significant efficiency bottlenecks on modern hardware due to unstructured computations. To address these challenges, we propose GeoRA (Geometry-Aware Low-Rank Adaptation), which exploits the anisotropic and compressible nature of RL update subspaces. GeoRA initializes adapters by extracting principal directions via Singular Value Decomposition (SVD) within a geometrically constrained subspace while freezing the residual components. This method preserves the pre-trained geometric structure and enables efficient GPU computation through dense operators. Experiments on Qwen and Llama demonstrate that GeoRA mitigates optimization bottlenecks caused by geometric misalignment. It consistently outperforms established low-rank baselines on key mathematical benchmarks, achieving state-of-the-art (SOTA) results. Moreover, GeoRA shows superior generalization and resilience to catastrophic forgetting in out-of-domain tasks.

Figures (6)

• Figure 1: Comparison of adapter initialization and forward architectures. LoRA applies low-rank adaptation on the original weight matrix $W$ with standard initialization, while PiSSA initializes adapters from the principal components of $W$. In contrast, GeoRA initializes from a geometry-constrained matrix $W_{\text{Geo}}$ (a different adaptation target than $W$). Its forward pass incorporates a frozen Residual Matrix in parallel with the trainable adapter to act as a stability anchor for principal components.
• Figure 2: Geometric Prior Construction via Masking. The process of generating $M_{\text{Geo}}$ by combining Spectral Priors (low-curvature regions) and Euclidean Priors (high-plasticity near-zero weights). The resulting $W_{\text{Geo}}$ isolates the most stable parameters for RL-native adaptation.
• Figure 3: Training dynamics of Qwen3-8B on the AIME benchmark (average of 2024 and 2025). GeoRA remains consistently top-performing throughout training.
• Figure 4: Performance comparison across different learning rates. Unlike baselines, GeoRA demonstrates superior stability and robust convergence even at higher learning rates.
• Figure 5: Training Stability and Constraint Adherence. Under aggressive learning rates ($5 \times 10^{-5}$), GeoRA demonstrates superior robustness.