Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Algorithm Design for Online Meta-Learning with Task Boundary Detection

Daouda Sow, Sen Lin, Yingbin Liang, Junshan Zhang

TL;DR

This paper tackles online meta-learning under non-stationary task distributions with unknown task boundaries. It introduces LEEDS, a memory-efficient algorithm that uses simple detectors for task switches and distribution shifts to selectively reuse the best task model and update the meta-model based on current data, achieving sublinear task-averaged regret. Theoretical guarantees quantify the impact of task similarity and boundary-detection uncertainty, while extensive experiments across three benchmarks show LEEDS outperforms strong baselines, especially on OOD tasks, and remains robust to threshold choices. The work advances practical online meta-learning by enabling fast adaptation, continual knowledge retention, and light memory requirements in dynamic environments.

Abstract

Online meta-learning has recently emerged as a marriage between batch meta-learning and online learning, for achieving the capability of quick adaptation on new tasks in a lifelong manner. However, most existing approaches focus on the restrictive setting where the distribution of the online tasks remains fixed with known task boundaries. In this work, we relax these assumptions and propose a novel algorithm for task-agnostic online meta-learning in non-stationary environments. More specifically, we first propose two simple but effective detection mechanisms of task switches and distribution shift based on empirical observations, which serve as a key building block for more elegant online model updates in our algorithm: the task switch detection mechanism allows reusing of the best model available for the current task at hand, and the distribution shift detection mechanism differentiates the meta model update in order to preserve the knowledge for in-distribution tasks and quickly learn the new knowledge for out-of-distribution tasks. In particular, our online meta model updates are based only on the current data, which eliminates the need of storing previous data as required in most existing methods. We further show that a sublinear task-averaged regret can be achieved for our algorithm under mild conditions. Empirical studies on three different benchmarks clearly demonstrate the significant advantage of our algorithm over related baseline approaches.

Algorithm Design for Online Meta-Learning with Task Boundary Detection

TL;DR

This paper tackles online meta-learning under non-stationary task distributions with unknown task boundaries. It introduces LEEDS, a memory-efficient algorithm that uses simple detectors for task switches and distribution shifts to selectively reuse the best task model and update the meta-model based on current data, achieving sublinear task-averaged regret. Theoretical guarantees quantify the impact of task similarity and boundary-detection uncertainty, while extensive experiments across three benchmarks show LEEDS outperforms strong baselines, especially on OOD tasks, and remains robust to threshold choices. The work advances practical online meta-learning by enabling fast adaptation, continual knowledge retention, and light memory requirements in dynamic environments.

Abstract

Online meta-learning has recently emerged as a marriage between batch meta-learning and online learning, for achieving the capability of quick adaptation on new tasks in a lifelong manner. However, most existing approaches focus on the restrictive setting where the distribution of the online tasks remains fixed with known task boundaries. In this work, we relax these assumptions and propose a novel algorithm for task-agnostic online meta-learning in non-stationary environments. More specifically, we first propose two simple but effective detection mechanisms of task switches and distribution shift based on empirical observations, which serve as a key building block for more elegant online model updates in our algorithm: the task switch detection mechanism allows reusing of the best model available for the current task at hand, and the distribution shift detection mechanism differentiates the meta model update in order to preserve the knowledge for in-distribution tasks and quickly learn the new knowledge for out-of-distribution tasks. In particular, our online meta model updates are based only on the current data, which eliminates the need of storing previous data as required in most existing methods. We further show that a sublinear task-averaged regret can be achieved for our algorithm under mild conditions. Empirical studies on three different benchmarks clearly demonstrate the significant advantage of our algorithm over related baseline approaches.
Paper Structure (20 sections, 1 theorem, 30 equations, 10 figures, 4 tables, 1 algorithm)

This paper contains 20 sections, 1 theorem, 30 equations, 10 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Problem Formulation
  3. Proposed Algorithm under Distribution Shifts
  4. Theoretical Results
  5. Experiments
  6. Main results
  7. Ablation Studies
  8. Conclusions
  9. More Related Work about Continual Learning
  10. More Experimental Results
  11. Accuracy tables with variances for Tiered-ImageNet and Synbols datasets
  12. Sensitivity to thresholds $\ell$ and $\tau$ and temperature $\delta$
  13. Further Descriptions about Datasets and Baseline Methods
  14. Datasets
  15. Baseline Methods
  16. ...and 5 more sections

Key Result

Theorem 4.5

Suppose Assumptions assum:struct,assum:adapt, assum:lsmooth, assum:lthr hold. Let $R=\sum_{t=1}^T K_t$ be the total number of online rounds, and $S=c\log R$ be the number of data points used for adaptation at each step, where $c$ is some positive constant. Then the expected TAR is bounded as where the expectation is taken over the task-boundary detection uncertainty. $\sigma_*^2 = \frac{1}{T} \su

Figures (10)

  • Figure 1: Left plot: Variations of the online loss for a pre-trained meta model using MAML which is deployed for online learning. Red dot at 0 means no task switch at that time, and at 1 means the task switched at that time. Right table: Comparison of the memory requirements among different methods. $T$ is number of online rounds and $p \in (0,1)$ is non-stationarity level.
  • Figure 2: Left: LEEDS under different $p$. Center: LEEDS with and without domain adaptation. Right: Task boundaries detection on Tiered-ImageNet (TI) and Synbols (SB).
  • Figure 3: Performance of LEEDS for different values of the threshold $\ell$. Left plot: Performance on all encountered domains during online learning. Right table: Task boundaries detection for different values of $\ell$. Experiments are conducted on the Omniglot-MNIST-FashionMNIST benchmark.
  • Figure 4: Performance of LEEDS for: (a) different values of the energy threshold $\tau$ and (b) different scales of the temperature $\delta$. For both plots we report the performance on all encountered domains during online learning. Experiments are conducted on the Omniglot-MNIST-FashionMNIST benchmark.
  • Figure 5: Online evaluations in each of the encountered domains during online learning phase for the Omniglot-MNIST-FashionMNIST benchmark. First column corresponds to non-stationarity level $p=0.9$. In second column $p=0.75$. LEEDS is the only method that is able to preserve pre-training knowledge while substantially increasing performance in OOD domains. Legend in first plot only.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Theorem 4.5