Unsupervised Replay Strategies for Continual Learning with Limited Data

TL;DR

This study discovered that introducing a sleep phase significantly enhanced accuracy in models trained with limited data, and highlighted the multifaceted role of sleep replay in augmenting learning efficiency and facilitating continual learning in ANNs.

Abstract

Artificial neural networks (ANNs) show limited performance with scarce or imbalanced training data and face challenges with continuous learning, such as forgetting previously learned data after new tasks training. In contrast, the human brain can learn continuously and from just a few examples. This research explores the impact of 'sleep', an unsupervised phase incorporating stochastic activation with local Hebbian learning rules, on ANNs trained incrementally with limited and imbalanced datasets, specifically MNIST and Fashion MNIST. We discovered that introducing a sleep phase significantly enhanced accuracy in models trained with limited data. When a few tasks were trained sequentially, sleep replay not only rescued previously learned information that had been catastrophically forgetting following new task training but often enhanced performance in prior tasks, especially those trained with limited data. This study highlights the multifaceted role of sleep replay in augmenting learning efficiency and facilitating continual learning in ANNs.

Figures (9)

• Figure 1: Accuracy on MNIST (A) and FMNIST (B) with mean (lines) and standard deviation (error bars) across 10 trials. X-axis - log of the relative amount of data used for training (e.g., 0.01=1% of data). Blue - baseline (after ANN training); Orange - baseline + sleep; Green - baseline + sleep + fine-tuning. Note significant gain in accuracy after the sleep phase on low data. The sleep phase reduced performance on high data but was largely recovered by fine-tuning.
• Figure 2: Confusion matrices before and after sleep for MNIST dataset. A 3% subset of the overall MNIST dataset was used in training. The value in each cell indicates the fraction of images of a given true label that were classified as a given predicted label by the model. (A) - before SRC, (B) - after SRC.
• Figure 3: Confusion matrices before and after sleep for FMNIST dataset. A 3% subset of the overall FMNIST dataset was used in training. The value in each cell indicates the fraction of images of a given true label that were classified as a given predicted label by the model. (A) - before SRC, (B) - after SRC.
• Figure 4: Imbalanced class accuracy improvement due to sleep. Each row shows experiments with data reduction for one specific class (shown on the left), with the percentage of reduction shown on the horizontal axis. Each cell shows the class-wise accuracy of the underrepresented class before sleep (top value) and after sleep (bottom value). The color map is based on the change in accuracy, $\Delta=$ After Sleep - Before Sleep. Reds indicate a positive difference (improvement), while blues indicate a negative difference (drop in accuracy). Note, many red squares showing class-wise improvement with only a few blue squares showing class-wise performance loss.
• Figure 5: Heatmaps showing accuracy changes after each phase of sequential learning and SRC (columns) on MNIST (A) and FMNIST (B). In each subplot, X-axis represents amount of T2 training data and Y-axis - amount of T1 training data. The rows show accuracy on T1, T2 and the mean accuracy.