Nested Learning: The Illusion of Deep Learning Architectures

TL;DR

This paper proposes Nested Learning (NL), a brain-inspired paradigm that treats a neural model and its training as a hierarchy of nested optimization problems, each with its own context and update frequency. It reframes architectures and optimizers as interconnected associative memories and introduces Continuum Memory Systems (CMS) to store memories across multiple timescales, along with Hope, a self-modifying learning module that combines CMS with self-referential dynamics. Key contributions include expressive memory-augmented optimizers (e.g., Delta Gradient Descent, multi-scale momentum), a formal NSAM framework, and empirical demonstrations showing improved continual learning and long-context understanding across language modeling, QA, and translation tasks. The work offers a unified lens to reinterpret existing models and outlines a practical path toward more expressive, adaptable systems with potential impact on continual learning and large-context reasoning in AI systems.

Abstract

Despite the recent progresses, particularly in developing Language Models, there are fundamental challenges and unanswered questions about how such models can continually learn/memorize, self-improve, and find effective solutions. In this paper, we present a new learning paradigm, called Nested Learning (NL), that coherently represents a machine learning model with a set of nested, multi-level, and/or parallel optimization problems, each of which with its own context flow. Through the lenses of NL, existing deep learning methods learns from data through compressing their own context flow, and in-context learning naturally emerges in large models. NL suggests a philosophy to design more expressive learning algorithms with more levels, resulting in higher-order in-context learning and potentially unlocking effective continual learning capabilities. We advocate for NL by presenting three core contributions: (1) Expressive Optimizers: We show that known gradient-based optimizers, such as Adam, SGD with Momentum, etc., are in fact associative memory modules that aim to compress the gradients' information (by gradient descent). Building on this insight, we present other more expressive optimizers with deep memory and/or more powerful learning rules; (2) Self-Modifying Learning Module: Taking advantage of NL's insights on learning algorithms, we present a sequence model that learns how to modify itself by learning its own update algorithm; and (3) Continuum Memory System: We present a new formulation for memory system that generalizes the traditional viewpoint of long/short-term memory. Combining our self-modifying sequence model with the continuum memory system, we present a continual learning module, called Hope, showing promising results in language modeling, knowledge incorporation, and few-shot generalization tasks, continual learning, and long-context reasoning tasks.

Figures (13)

• Figure 1: The uniform and reusable structure as well as multi time scale update in the brain are the key components to unlock the continual learning in humans. Nested Learning (NL) allows for multi time-scale update for each component of the brain, while showing that well-known architectures such as Transformers are in fact linear layers with different frequency updates.
• Figure 2: Nested Learning Paradigm that represent a machine learning model and its training procedure as a set of nested optimization problems. (Left) An example of Hybrid architecture. While deep learning perspective, as the flattened image of NL, does not provide insight about the depth of computation in the blocks, NL transparently represent all the inner gradient flows. (Right) A Neural Learning Module: A computational model that learns how to compress its own context flow. For example, the first level corresponds to the model's most outer-loop training, often refer to as "pre-training" step.
• Figure 3: An example of comparing a FFN (e.g., MLP) with linear attention in a Transformer-based backbone, optimizing with gradient descent. The red components are blocks in the first level (with frequency 1), while blue components are blocks in the second level (frequency $L$). Linear attention with learnable initial memory state (referred to as Linear Attention++) is the same as an MLP layer but with in-context learning ability and adaptation to the input sequence.
• Figure 4: Optimization of function $\boldsymbol{\psi}(r, \theta)$ with standard momentum and our delta momentum.
• Figure 5: A comparison of Hope architectural backbone with Transformers (Normalization and potential data-dependent components are removed for the sake of clarity).

Theorems & Definitions (7)

• Definition 1: Associative Memory
• Definition 2: Update Frequency
• Definition 3: Nested System
• Definition 4: Nested System of Associative Memories
• Definition 5: Generalized Gradient Descent (GGD) Learning Rule
• Definition 6: (Generalized) Nested System
• Definition 7: (Generalized) Nested System of Associative Memories