Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IBCL: Zero-shot Model Generation under Stability-Plasticity Trade-offs

Pengyuan Lu, Michele Caprio, Eric Eaton, Insup Lee

TL;DR

CLuST seeks models tailored to explicit stability-plasticity preferences. IBCL addresses this by maintaining a finitely generated credal set of task posteriors and generating Pareto-optimal models for any given preference in zero-shot via convex combinations and HDR-based regions. The method achieves up to 44% average per-task accuracy and 45% peak accuracy gains over baselines, with negligible backward transfer, constant training overhead per task, and sublinear memory growth; it also extends to reinforcement learning with preserved Pareto front quality and reduced training cost. This approach enables scalable, user-specific trade-off handling in continual learning without retraining for each new preference, offering practical impact for large, multi-task systems and personalized AI experiences.

Abstract

Algorithms that balance the stability-plasticity trade-off are well studied in the Continual Learning literature. However, only a few focus on obtaining models for specified trade-off preferences. When solving the problem of continual learning under specific trade-offs (CLuST), state-of-the-art techniques leverage rehearsal-based learning, which requires retraining when a model corresponding to a new trade-off preference is requested. This is inefficient, since there potentially exists a significant number of different trade-offs, and a large number of models may be requested. As a response, we propose Imprecise Bayesian Continual Learning (IBCL), an algorithm that tackles CLuST efficiently. IBCL replaces retraining with a constant-time convex combination. Given a new task, IBCL (1) updates the knowledge base as a convex hull of model parameter distributions, and (2) generates one Pareto-optimal model per given trade-off via convex combination without additional training. That is, obtaining models corresponding to specified trade-offs via IBCL is zero-shot. Experiments whose baselines are current CLuST algorithms show that IBCL improves classification by at most 44% on average per task accuracy, and by 45% on peak per task accuracy while maintaining a near-zero to positive backward transfer, with memory overheads converging to constants. In addition, its training overhead, measured by the number of batch updates, remains constant at every task, regardless of the number of preferences requested. IBCL also improves multi-objective reinforcement learning tasks by maintaining the same Pareto front hypervolume, while significantly reducing the training cost. Details can be found at: https://github.com/ibcl-anon/ibcl.

IBCL: Zero-shot Model Generation under Stability-Plasticity Trade-offs

TL;DR

CLuST seeks models tailored to explicit stability-plasticity preferences. IBCL addresses this by maintaining a finitely generated credal set of task posteriors and generating Pareto-optimal models for any given preference in zero-shot via convex combinations and HDR-based regions. The method achieves up to 44% average per-task accuracy and 45% peak accuracy gains over baselines, with negligible backward transfer, constant training overhead per task, and sublinear memory growth; it also extends to reinforcement learning with preserved Pareto front quality and reduced training cost. This approach enables scalable, user-specific trade-off handling in continual learning without retraining for each new preference, offering practical impact for large, multi-task systems and personalized AI experiences.

Abstract

Algorithms that balance the stability-plasticity trade-off are well studied in the Continual Learning literature. However, only a few focus on obtaining models for specified trade-off preferences. When solving the problem of continual learning under specific trade-offs (CLuST), state-of-the-art techniques leverage rehearsal-based learning, which requires retraining when a model corresponding to a new trade-off preference is requested. This is inefficient, since there potentially exists a significant number of different trade-offs, and a large number of models may be requested. As a response, we propose Imprecise Bayesian Continual Learning (IBCL), an algorithm that tackles CLuST efficiently. IBCL replaces retraining with a constant-time convex combination. Given a new task, IBCL (1) updates the knowledge base as a convex hull of model parameter distributions, and (2) generates one Pareto-optimal model per given trade-off via convex combination without additional training. That is, obtaining models corresponding to specified trade-offs via IBCL is zero-shot. Experiments whose baselines are current CLuST algorithms show that IBCL improves classification by at most 44% on average per task accuracy, and by 45% on peak per task accuracy while maintaining a near-zero to positive backward transfer, with memory overheads converging to constants. In addition, its training overhead, measured by the number of batch updates, remains constant at every task, regardless of the number of preferences requested. IBCL also improves multi-objective reinforcement learning tasks by maintaining the same Pareto front hypervolume, while significantly reducing the training cost. Details can be found at: https://github.com/ibcl-anon/ibcl.
Paper Structure (35 sections, 4 theorems, 14 equations, 13 figures, 3 tables, 4 algorithms)

This paper contains 35 sections, 4 theorems, 14 equations, 13 figures, 3 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Imprecise Probability
  4. Continual Learning
  5. Stability-plasticity Trade-offs
  6. Continual Learning under Specific Trade-offs (CLuST)
  7. Formulating the CLuST Problem
  8. Assumptions
  9. Details of Assumption \ref{['assum:task_similarity']}
  10. Main Problem
  11. Imprecise Bayesian Continual Learning
  12. FGCS Knowledge Base Update
  13. Zero-shot Generation of User Preferred Models
  14. Overall IBCL Algorithm and Analysis
  15. Reinforcement Learning using IBCL
  16. ...and 20 more sections

Key Result

Proposition 4.1

Let $q_k^j$ be an extreme point posterior of $\mathcal{Q}_i$ learned from the $j$-th prior at task $k \in \{1,\ldots,i\}$. For any preference $\bar{w} = (w_1, \dots, w_i)^\top$ on tasks $\{1,\ldots,i\}$, there exists a probability vector $\bar{\beta} = (\beta_1^1, \dots, \beta_1^{m_1}, \dots, \beta_ In other words, selecting a precise distribution $\hat{q}_{\bar{w}}$ from $\mathcal{Q}_i$ is equiva

Figures (13)

  • Figure 1: A Bayesian view of a Pareto-optimal parameter distribution $q'$ and a non-Pareto-optimal parameter distribution $q"$.
  • Figure 2: The workflow of Imprecise Bayesian Continual Learning (IBCL). Here, we start from 1 prior, but in practice, there may be more than 1 to reduce epistemic uncertainty eyke.
  • Figure 3: The $0.25$-HDR for a Normal Mixture density. This is a replica of hyndman.
  • Figure 4: Results of 20 News Group (left column) and Split CIFAR-100 (right column).
  • Figure 5: Results of CelebA (left column) and Tiny ImageNet (right column).
  • ...and 8 more figures

Theorems & Definitions (13)

  • Definition 2.1: Finitely Generated Credal Set
  • Definition 2.2: Highest Density Region
  • Definition 2.3: Highest Density Region, Alternative
  • Definition 2.4: Bayesian Continual Learning
  • Definition 3.1: 2-Wasserstein Metric on $\Delta_{\mathcal{XY}}$
  • Definition 3.4: Stability-plasticity Trade-off Preferences over Tasks
  • Proposition 4.1: Selection Equivalence
  • Proposition 4.2: Probabilistic Pareto-optimality
  • Proposition A.1
  • proof
  • ...and 3 more