Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task

Huiping Zhuang, Di Fang, Kai Tong, Yuchen Liu, Ziqian Zeng, Xu Zhou, Cen Chen

TL;DR

The AEF-OCL solves the OCL problem by recursively calculating the analytical solution, ensuring an equalization between the continual learning and its joint-learning counterpart, and works without the need to save any used samples (i.e., exemplar-free).

Abstract

In autonomous driving, even a meticulously trained model can encounter failures when facing unfamiliar scenarios. One of these scenarios can be formulated as an online continual learning (OCL) problem. That is, data come in an online fashion, and models are updated according to these streaming data. Two major OCL challenges are catastrophic forgetting and data imbalance. To address these challenges, in this paper, we propose an Analytic Exemplar-Free Online Continual Learning algorithm (AEF-OCL). The AEF-OCL leverages analytic continual learning principles and employs ridge regression as a classifier for features extracted by a large backbone network. It solves the OCL problem by recursively calculating the analytical solution, ensuring an equalization between the continual learning and its joint-learning counterpart, and works without the need to save any used samples (i.e., exemplar-free). Additionally, we introduce a Pseudo-Features Generator (PFG) module that recursively estimates the mean and the variance of real features for each class. It over-samples offset pseudo-features from the same normal distribution as the real features, thereby addressing the data imbalance issue. Experimental results demonstrate that despite being an exemplar-free strategy, our method outperforms various methods on the autonomous driving SODA10M dataset. Source code is available at https://github.com/ZHUANGHP/Analytic-continual-learning.

Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task

TL;DR

The AEF-OCL solves the OCL problem by recursively calculating the analytical solution, ensuring an equalization between the continual learning and its joint-learning counterpart, and works without the need to save any used samples (i.e., exemplar-free).

Abstract

In autonomous driving, even a meticulously trained model can encounter failures when facing unfamiliar scenarios. One of these scenarios can be formulated as an online continual learning (OCL) problem. That is, data come in an online fashion, and models are updated according to these streaming data. Two major OCL challenges are catastrophic forgetting and data imbalance. To address these challenges, in this paper, we propose an Analytic Exemplar-Free Online Continual Learning algorithm (AEF-OCL). The AEF-OCL leverages analytic continual learning principles and employs ridge regression as a classifier for features extracted by a large backbone network. It solves the OCL problem by recursively calculating the analytical solution, ensuring an equalization between the continual learning and its joint-learning counterpart, and works without the need to save any used samples (i.e., exemplar-free). Additionally, we introduce a Pseudo-Features Generator (PFG) module that recursively estimates the mean and the variance of real features for each class. It over-samples offset pseudo-features from the same normal distribution as the real features, thereby addressing the data imbalance issue. Experimental results demonstrate that despite being an exemplar-free strategy, our method outperforms various methods on the autonomous driving SODA10M dataset. Source code is available at https://github.com/ZHUANGHP/Analytic-continual-learning.
Paper Structure (25 sections, 2 theorems, 20 equations, 6 figures, 1 table, 2 algorithms)

This paper contains 25 sections, 2 theorems, 20 equations, 6 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. The SODA10M dataset
  4. Continual Learning Methods
  5. Replay-based CL
  6. Exemplar-free CL
  7. Online Continual Learning
  8. CL with Large Pre-trained Models
  9. Data Imbalanced Continual Learning
  10. Proposed Method
  11. Overview
  12. Feature Extraction
  13. Ridge Regression Classifier
  14. Continual Learning
  15. Pseudo-Features Generation
  16. ...and 10 more sections

Key Result

Theorem 1

The calculation of the regularized feature autocorrelation matrix at task $k$, $\bm{R}_k = (\sum_{i=1}^{k}\bm{X}_{i}^\top \bm{X}_{i} + \gamma \bm{I})^{-1}$ is identical to its recursive form where $\bm{R}_0 = \frac{1}{\gamma}\bm{I}$.

Figures (6)

  • Figure 1: The training process of our proposed method includes: (a) a large universal frozen pre-trained backbone such as a ViT without its classification head; (b) a pseudo-features generator that estimates the mean and the variance of features recursively and generates the offset pseudo-features in an estimated normal distribution to balance the training data; (c) an iterative ridge regression classifier that iteratively updates its weight with real features only; (d) a balanced ridge regression classifier for inference that updates its weight from the iterative classifier using offset pseudo-features generated at each task.
  • Figure 2: The number of training samples of each class.
  • Figure 3: Distributions of the first element of features of different classes.
  • Figure 4: Distributions of the first 6 elements of features of the Car class.
  • Figure 5: Different update strategies on different regularization weight.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Theorem 2
  • proof