Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Representation Stability in a Minimal Continual Learning Agent

Vishnu Subramanian

TL;DR

This work studies a minimal continual learning agent designed to isolate representational dynamics from architectural complexity and optimization objectives, which establishes a transparent empirical baseline for studying representational accumulation and adaptation in continual learning systems.

Abstract

Continual learning systems are increasingly deployed in environments where retraining or reset is infeasible, yet many approaches emphasize task performance rather than the evolution of internal representations over time. In this work, we study a minimal continual learning agent designed to isolate representational dynamics from architectural complexity and optimization objectives. The agent maintains a persistent state vector across executions and incrementally updates it as new textual data is introduced. We quantify representational change using cosine similarity between successive normalized state vectors and define a stability metric over time intervals. Longitudinal experiments across eight executions reveal a transition from an initial plastic regime to a stable representational regime under consistent input. A deliberately introduced semantic perturbation produces a bounded decrease in similarity, followed by recovery and restabilization under subsequent coherent input. These results demonstrate that meaningful stability plasticity tradeoffs can emerge in a minimal, stateful learning system without explicit regularization, replay, or architectural complexity. The work establishes a transparent empirical baseline for studying representational accumulation and adaptation in continual learning systems.

Representation Stability in a Minimal Continual Learning Agent

TL;DR

This work studies a minimal continual learning agent designed to isolate representational dynamics from architectural complexity and optimization objectives, which establishes a transparent empirical baseline for studying representational accumulation and adaptation in continual learning systems.

Abstract

Continual learning systems are increasingly deployed in environments where retraining or reset is infeasible, yet many approaches emphasize task performance rather than the evolution of internal representations over time. In this work, we study a minimal continual learning agent designed to isolate representational dynamics from architectural complexity and optimization objectives. The agent maintains a persistent state vector across executions and incrementally updates it as new textual data is introduced. We quantify representational change using cosine similarity between successive normalized state vectors and define a stability metric over time intervals. Longitudinal experiments across eight executions reveal a transition from an initial plastic regime to a stable representational regime under consistent input. A deliberately introduced semantic perturbation produces a bounded decrease in similarity, followed by recovery and restabilization under subsequent coherent input. These results demonstrate that meaningful stability plasticity tradeoffs can emerge in a minimal, stateful learning system without explicit regularization, replay, or architectural complexity. The work establishes a transparent empirical baseline for studying representational accumulation and adaptation in continual learning systems.
Paper Structure (13 sections, 2 equations, 1 figure, 1 table)

This paper contains 13 sections, 2 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Minimal Continual Learning Agent
  4. Learning Loop
  5. Representation and State Definition
  6. Representational Similarity Metric
  7. Results
  8. Longitudinal Representation Dynamics
  9. Response to Semantic Perturbation
  10. Vocabulary Growth and State Expansion
  11. Discussion
  12. Limitations and Future Work
  13. Conclusion

Figures (1)

  • Figure 1: Cosine similarity between successive normalized internal state vectors across eight continual executions. Early runs exhibit rapid representational change, followed by stabilization under consistent input. A semantically orthogonal perturbation at Day 5 induces a bounded decrease in similarity to 0.8957, after which the representation recovers and re-stabilizes under subsequent coherent input.