Continual Learning, Not Training: Online Adaptation For Agents

TL;DR

The paper tackles the mismatch between traditional continual learning and real-time deployment by proposing a system-centric, gradient-free framework called ATLAS that decouples reasoning from execution and uses a Persistent Learning Memory to enable inference-time adaptation. Through a dual-agent Teacher–Student architecture guided by a-principled reward signals, ATLAS achieves efficient performance with significantly reduced token usage while matching or surpassing larger models on a cyber-threat benchmark. The approach yields transferable, causally annotated traces that can train explicit world models, and empirical results on ExCyTIn-Bench Incident #5 demonstrate substantial gains in both efficacy and efficiency, with cross-incident transfer showing generalization. Overall, system-centric continual learning offers a practical, deployable path toward adaptive AI that learns from use and provides audit-friendly traces for downstream modeling.

Abstract

Continual Learning (CL) methods have traditionally focused on mitigating catastrophic forgetting through gradient-based retraining, an approach ill-suited for deployed agents that must adapt in real time. We introduce our Adaptive Teaching and Learning System (ATLAS), a dual-agent architecture that decouples reasoning (Teacher) from execution (Student) and incorporates a persistent learning memory that stores distilled guidance from experience. This informs the orchestration layer, enabling the system to dynamically adjust its operational strategies, such as supervision level or initial plan selection, at inference time. In doing so, ATLAS achieves gradient-free continual learning, shifting the locus of adaptation from model parameters to system-level orchestration. We formulate this as a system-centric paradigm for continual learning, where the objective is adaptive efficiency: maximizing task success while minimizing computational cost through inference-time orchestration rather than parameter updates. Evaluated on Microsoft's ExCyTIn-Bench, an open-source benchmark simulating complex cyberthreat investigation, ATLAS achieves 54.1% success with GPT-5-mini as its Student, outperforming the larger GPT-5 (High) by 13% while reducing cost by 86%. Cross-incident validation demonstrates generalization: frozen pamphlets from Incident #5 improve accuracy from 28% to 41% with zero retraining, while shifting output composition from verbose exploration to structured reasoning. Together, these findings establish gradient-free continual learning as a viable path toward adaptive, deployable AI systems and provide causally annotated traces valuable for training explicit world models.

Figures (4)

• Figure 1: ATLAS System Architecture for gradient-free adaptation. An Orchestrator manages Teacher-Student interactions during the execution loop. Post-run, learning is evaluated, synthesized, and stored in Persistent Learning Memory (PLM) to guide future inference-time decisions.
• Figure 2: Continual learning progression across 98 tasks on ExCyTIn-Bench Incident #5. The learning journey unfolds in three phases: (1) Domain Acquisition (tasks 1-25) establishes foundational cyber investigation skills; (2) Task Adaptation (tasks 26-60) navigates diverse challenge variations; (3) Skill Mastery (tasks 61-98) achieves peak efficiency through consolidated learnings. Green trend line shows progressive improvement from 32.8% to 53.9% efficiency gain, demonstrating 21.1 percentage point improvement through inference-time continual learning.
• Figure 3: Performance and cost comparison on ExCyTIn-Bench (Incident #5). ATLAS (blue) achieves higher success (54.1%) at a lower cost ( $0.024/task) than the stronger GPT-5 (High) baseline (48.0%, $0.174/task).
• Figure 4: Cross-incident transfer performance from Incident #5 to Incident #55. Using learned pamphlets from Incident #5, ATLAS improved success rate from 28% to 41% (+46%) with no additional training or real-time guidance.