Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GEM-Style Constraints for PEFT with Dual Gradient Projection in LoRA

Brian Tekmen, Jason Yin, Qianqian Tong

TL;DR

The paper addresses continual learning for large language models under parameter-efficient fine-tuning by restricting non-interference constraints to the LoRA adapter subspace. It introduces I-GEM, a fixed-budget dual PGD method that approximately enforces GEM-like non-interference without solving a full QP, achieving on-device, GPU-resident updates with $O(K t d_\phi)$ cost per projection. Empirically, I-GEM matches exact GEM in average accuracy (within about $0.04$ points) on a domain-drift AG News setup while reducing projection time by roughly $10^3\times$ and remaining more accurate than A-GEM by about $1.4$ points. These results demonstrate a practical pathway for GEM-style continual learning at LLM scale using LoRA, enabling stable, scalable fine-tuning with modest overhead.

Abstract

Full fine-tuning of Large Language Models (LLMs) is computationally costly, motivating Continual Learning (CL) approaches that utilize parameter-efficient adapters. We revisit Gradient Episodic Memory (GEM) within the Low-Rank Adapter (LoRA) subspace and introduce I-GEM: a fixed-budget, GPU-resident dual projected-gradient approximation to GEM's quadratic projection. By constraining non-interference solely within the adapter parameters, I-GEM preserves GEM-like stability with orders-of-magnitude lower mean projection overhead. On a 3-task AG News split with induced domain drift, using GPT-2 (355M) and LoRA ($r=8$), I-GEM matches GEM's average accuracy (within $\sim\!0.04$ pts) and outperforms A-GEM by $\sim\!1.4$ pts. Crucially, it reduces projection time vs.\ GEM by a factor of $\sim\!10^3$. These results suggest that applying GEM constraints in the LoRA subspace is a practical pathway for continual learning at the LLM scale.

GEM-Style Constraints for PEFT with Dual Gradient Projection in LoRA

TL;DR

The paper addresses continual learning for large language models under parameter-efficient fine-tuning by restricting non-interference constraints to the LoRA adapter subspace. It introduces I-GEM, a fixed-budget dual PGD method that approximately enforces GEM-like non-interference without solving a full QP, achieving on-device, GPU-resident updates with cost per projection. Empirically, I-GEM matches exact GEM in average accuracy (within about points) on a domain-drift AG News setup while reducing projection time by roughly and remaining more accurate than A-GEM by about points. These results demonstrate a practical pathway for GEM-style continual learning at LLM scale using LoRA, enabling stable, scalable fine-tuning with modest overhead.

Abstract

Full fine-tuning of Large Language Models (LLMs) is computationally costly, motivating Continual Learning (CL) approaches that utilize parameter-efficient adapters. We revisit Gradient Episodic Memory (GEM) within the Low-Rank Adapter (LoRA) subspace and introduce I-GEM: a fixed-budget, GPU-resident dual projected-gradient approximation to GEM's quadratic projection. By constraining non-interference solely within the adapter parameters, I-GEM preserves GEM-like stability with orders-of-magnitude lower mean projection overhead. On a 3-task AG News split with induced domain drift, using GPT-2 (355M) and LoRA (), I-GEM matches GEM's average accuracy (within pts) and outperforms A-GEM by pts. Crucially, it reduces projection time vs.\ GEM by a factor of . These results suggest that applying GEM constraints in the LoRA subspace is a practical pathway for continual learning at the LLM scale.
Paper Structure (28 sections, 26 equations, 4 figures, 2 tables, 2 algorithms)

This paper contains 28 sections, 26 equations, 4 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Continual Learning
  4. Replay and gradient constraints
  5. PEFT with LoRA
  6. GEM in the LoRA subspace
  7. Dual form and fixed-budget projection
  8. Practical constraints and scope
  9. Methods
  10. Revisit GEM-family Methods
  11. I-GEM: Fixed-budget Dual PGD
  12. Feasibility of GEM in the LoRA subspace
  13. Implementation Details
  14. Experiments
  15. Datasets and protocol
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Final Average Accuracy at the end of training.
  • Figure 2: Per-task accuracy vs. training step on AG News ($T{=}3$).
  • Figure 3: CL metrics (AvgAcc, BWT, FWT) at the end of training.
  • Figure 4: Mean projection overhead (log scale; seconds) measured with synchronized CUDA events.