Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sample Weight Estimation Using Meta-Updates for Online Continual Learning

Hamed Hemati, Damian Borth

TL;DR

This paper tackles online continual learning by making per-sample weights in the loss learnable via online meta-learning. Using a replay buffer as a distribution proxy, OMSI performs inner updates on a combined mini-batch, then updates sample weights through a meta-objective before applying the final model update. The approach yields improved retained accuracy across standard CL benchmarks, especially under label noise, while remaining more efficient than fully online meta-replay methods. The results demonstrate the practical potential of self-adaptive CL with learnable loss weighting, and point to future work on larger data, richer meta-objectives, and unrolling optimizations.

Abstract

The loss function plays an important role in optimizing the performance of a learning system. A crucial aspect of the loss function is the assignment of sample weights within a mini-batch during loss computation. In the context of continual learning (CL), most existing strategies uniformly treat samples when calculating the loss value, thereby assigning equal weights to each sample. While this approach can be effective in certain standard benchmarks, its optimal effectiveness, particularly in more complex scenarios, remains underexplored. This is particularly pertinent in training "in the wild," such as with self-training, where labeling is automated using a reference model. This paper introduces the Online Meta-learning for Sample Importance (OMSI) strategy that approximates sample weights for a mini-batch in an online CL stream using an inner- and meta-update mechanism. This is done by first estimating sample weight parameters for each sample in the mini-batch, then, updating the model with the adapted sample weights. We evaluate OMSI in two distinct experimental settings. First, we show that OMSI enhances both learning and retained accuracy in a controlled noisy-labeled data stream. Then, we test the strategy in three standard benchmarks and compare it with other popular replay-based strategies. This research aims to foster the ongoing exploration in the area of self-adaptive CL.

Sample Weight Estimation Using Meta-Updates for Online Continual Learning

TL;DR

This paper tackles online continual learning by making per-sample weights in the loss learnable via online meta-learning. Using a replay buffer as a distribution proxy, OMSI performs inner updates on a combined mini-batch, then updates sample weights through a meta-objective before applying the final model update. The approach yields improved retained accuracy across standard CL benchmarks, especially under label noise, while remaining more efficient than fully online meta-replay methods. The results demonstrate the practical potential of self-adaptive CL with learnable loss weighting, and point to future work on larger data, richer meta-objectives, and unrolling optimizations.

Abstract

The loss function plays an important role in optimizing the performance of a learning system. A crucial aspect of the loss function is the assignment of sample weights within a mini-batch during loss computation. In the context of continual learning (CL), most existing strategies uniformly treat samples when calculating the loss value, thereby assigning equal weights to each sample. While this approach can be effective in certain standard benchmarks, its optimal effectiveness, particularly in more complex scenarios, remains underexplored. This is particularly pertinent in training "in the wild," such as with self-training, where labeling is automated using a reference model. This paper introduces the Online Meta-learning for Sample Importance (OMSI) strategy that approximates sample weights for a mini-batch in an online CL stream using an inner- and meta-update mechanism. This is done by first estimating sample weight parameters for each sample in the mini-batch, then, updating the model with the adapted sample weights. We evaluate OMSI in two distinct experimental settings. First, we show that OMSI enhances both learning and retained accuracy in a controlled noisy-labeled data stream. Then, we test the strategy in three standard benchmarks and compare it with other popular replay-based strategies. This research aims to foster the ongoing exploration in the area of self-adaptive CL.
Paper Structure (17 sections, 3 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 17 sections, 3 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Continual Learning with Experience Replay
  4. Continual Learning with Meta-Updates
  5. Adaptive Learning in Noisy Streams
  6. Preliminaries
  7. Online Continual Learning
  8. Hyperparameters as Meta-Parameters
  9. Method
  10. Analysis via Controlled Experiments
  11. Varying the Sample Weight Update Factor
  12. Number of Inner Updates
  13. Fraction of Noisy Data
  14. Experiments
  15. Experimental Setting
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Schematic of the proposed strategy. (Top) In the first step, the model receives a mini-batch from the stream. After initializing the meta-parameters $\mathbf{w}$, the inner updates are applied to compute the meta-gradients. (Bottom) The sample weights are updated using the meta-gradients to adapt to the current set of samples in the mini-batch.
  • Figure 2: Combined figure showing (a) the 3D visualization of normalized sample weights (meta-parameters) and (b) the effect of increasing the alpha factor.
  • Figure 3: Effect of the number of inner steps on the LA and RA metrics. Increasing the number of inner updates to $2$ or higher does not necessarily result in a significant performance gain.
  • Figure 4: Retained accuracy for all strategies in the Meta Album experiment. Each mini-batch is observed only once.