GRASP LoRA: GRPO Guided Adapter Sparsity Policy for Cross Lingual Transfer

TL;DR

GRASP LoRA introduces a GRPO-guided sparsity policy that learns a single global prune ratio $p$ online to merge English and target-language LoRA adapters on a frozen backbone, replacing costly grid searches. By evaluating a micro development slice and updating $p$ via a Gaussian policy with mean anchoring and an entropy bonus, it achieves better cross-lingual transfer on XL-Sum and MLQA for Arabic and Chinese while substantially reducing runtime. The approach maintains adapter reuse and PEFT efficiency, and includes a final prune-and-fine-tune step once the optimal $p^*$ is identified. Overall, the method improves semantic faithfulness, content coverage, and QA quality, while reducing development-set requirements and enabling practical low-resource deployment.

Abstract

Parameter efficient fine tuning is a way to adapt LLMs to new languages when compute or data are limited, yet adapter pipelines usually choose a global prune ratio by grid search. This practice is computationally expensive and development set intensive, since it repeats training, freezes sparsity, and misses fractional optima. We introduce GRASP LoRA (GRPO Guided Adapter Sparsity Policy), which treats global sparsity as a learnable control variable. A GRPO controller interleaves with training, periodically probing candidate prune ratios on a small micro development set and updating a single global prune ratio online from its reward signal. It operates on merged source and target LoRA adapters on a frozen backbone and replaces grid search with one controller run that learns a prune ratio, followed by a single final merge and prune fine tuning run with pruning fixed to that ratio. On cross lingual transfer from English into Arabic and Chinese, including XL-Sum summarization and MLQA extractive question answering with Llama 3 8B, GRASP LoRA improves semantic faithfulness, content coverage, and answer quality over strong target only and merge and prune baselines. It reduces end to end runtime by multiple times relative to grid search, lowers reliance on large development sets, and makes adapter reuse practical for low resource deployment.

Figures (3)

• Figure 1: GRASP LoRA pipeline in three phases: merge source and target LoRA adapters on a frozen backbone, learn prune ratio $p^\star$ from micro dev feedback with a GRPO controller, then prune once at $p^\star$ and fine tune on target data to produce the final target task model.
• Figure 2: Rolling mean $\pm$ one standard deviation of the committed prune ratio $p_{\text{curr}}$ on Arabic XL-Sum, using a 10-sample window over logged controller records. Narrower shaded bands indicate more stable $p_{\text{curr}}$ trajectories, while wider bands indicate higher variability.
• Figure 3: Qualitative examples on Arabic transfer. (a) XL-Sum summarization: GRASP LoRA at the learned prune ratio ($p^\star{=}67.49\%$) preserves the key factual detail that Tripoli is the capital of Libya, while LoRA, LoRAHub, and the best grid searched prune ratio omit it. (b) MLQA extractive QA: GRASP LoRA at $p^\star{=}48.97\%$ predicts the correct answer year (2013), while the baselines shown here predict incorrect years. All outputs are generated in Arabic; we show English glosses (approximate translations) under the Arabic text for readability.