Active perception and disentangled representations allow continual, episodic zero and few-shot learning

TL;DR

A Complementary Learning System in which the fast learner entirely foregoes generalization in exchange for continual zero-shot and few-shot learning is described, demonstrating that fast, context-driven reasoning can coexist with slow, structured generalization, providing a pathway for robust continual learning.

Abstract

Generalization is often regarded as an essential property of machine learning systems. However, perhaps not every component of a system needs to generalize. Training models for generalization typically produces entangled representations at the boundaries of entities or classes, which can lead to destructive interference when rapid, high-magnitude updates are required for continual or few-shot learning. Techniques for fast learning with non-interfering representations exist, but they generally fail to generalize. Here, we describe a Complementary Learning System (CLS) in which the fast learner entirely foregoes generalization in exchange for continual zero-shot and few-shot learning. Unlike most CLS approaches, which use episodic memory primarily for replay and consolidation, our fast, disentangled learner operates as a parallel reasoning system. The fast learner can overcome observation variability and uncertainty by leveraging a conventional slow, statistical learner within an active perception system: A contextual bias provided by the fast learner induces the slow learner to encode novel stimuli in familiar, generalized terms, enabling zero-shot and few-shot learning. This architecture demonstrates that fast, context-driven reasoning can coexist with slow, structured generalization, providing a pathway for robust continual learning.

Figures (7)

• Figure 1: A disentangled, fast-learning episodic RL model which entirely delegates generalization to a slow, statistical long-term memory (LTM) avoids the limitations of conventional statistical learning. Further explanation of these results are provided in section \ref{['sec:results']}.
• Figure 2: Active perception within a Complementary Learning System allows the STM to learn rapidly without needing to generalize. Generalization occurs within the LTM, which is manipulated by the STM to emit stable, familiar representations relevant to the task. Using the generalization ability within the LTM, the STM can even immediately reason about novel stimuli if they are expressed in terms the STM can already understand.
• Figure 3: Each episode represents an encounter with an object. The agent must decide how to respond to the object, potentially resulting in a reward. The action space available to the STM includes a set of perceptual queries to the LTM, which will attempt to make generalized inferences about the stimulus.
• Figure 4: Effect of cumulative few-shot training exclusively on new data with no further exposure to the original data. "Entangled" models trained with stochastic gradient descent rapidly lose performance on the original data even before learning the new data. The proposed disentangled memory maintains performance (mean reward per step) on original data while rapidly learning the new data. Rewards only occur at the end of an episode, which may take up to 10 steps. Mean rewards of approximately 0.1 and 0.08 represent an optimal policy on the original and new data respectively.
• Figure 5: An interesting phenomenon during episodic few-shot learning, illustrated by the result of just one run. The model appears to forget the original data during learning of new encounters, producing a characteristic V-shaped drop in performance on the original data. However, this is actually caused by changes in policy during exploration of the new data. After discovery of the optimal policy for the new data, performance rebounds to its prior level on the original data, showing that nothing was forgotten after all.