Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-Time Procedural Learning From Experience for AI Agents

Dasheng Bi, Yubin Hu, Mohammed N. Nasir

TL;DR

Real-world AI agents must acquire procedural knowledge after deployment in dynamic environments. The paper introduces PRAXIS, a lightweight state-dependent procedural memory that stores local interaction traces and retrieves them by jointly matching environmental and internal states, integrating it into the Altrina web agent. Evaluated on the REAL web-browsing benchmark, PRAXIS yields consistent improvements in accuracy, reliability, and efficiency across multiple backbone models and shows preliminary generalization to unseen tasks. The results demonstrate that state-dependent procedural recall provides reusable, context-grounded priors that enhance robust, adaptive web automation and potentially broader agentic applications.

Abstract

Learning how to do things from trial and error in real time is a hallmark of biological intelligence, yet most LLM-based agents lack mechanisms to acquire procedural knowledge after deployment. We propose Procedural Recall for Agents with eXperiences Indexed by State (PRAXIS), a lightweight post-training learning mechanism that stores the consequences of actions and retrieves them by jointly matching environmental and internal states of past episodes to the current state. PRAXIS augments agentic action selection with retrieved state-action-result exemplars that are generated in real time. When evaluated on the REAL web browsing benchmark, PRAXIS improves task completion accuracy, reliability, and cost efficiency across different foundation model backbones, and shows preliminary generalization to unseen tasks in similar environments. These results demonstrate that PRAXIS enables the practical adoption of AI agents in fast-evolving stateful environments by helping them learn new procedures effectively.

Real-Time Procedural Learning From Experience for AI Agents

TL;DR

Real-world AI agents must acquire procedural knowledge after deployment in dynamic environments. The paper introduces PRAXIS, a lightweight state-dependent procedural memory that stores local interaction traces and retrieves them by jointly matching environmental and internal states, integrating it into the Altrina web agent. Evaluated on the REAL web-browsing benchmark, PRAXIS yields consistent improvements in accuracy, reliability, and efficiency across multiple backbone models and shows preliminary generalization to unseen tasks. The results demonstrate that state-dependent procedural recall provides reusable, context-grounded priors that enhance robust, adaptive web automation and potentially broader agentic applications.

Abstract

Learning how to do things from trial and error in real time is a hallmark of biological intelligence, yet most LLM-based agents lack mechanisms to acquire procedural knowledge after deployment. We propose Procedural Recall for Agents with eXperiences Indexed by State (PRAXIS), a lightweight post-training learning mechanism that stores the consequences of actions and retrieves them by jointly matching environmental and internal states of past episodes to the current state. PRAXIS augments agentic action selection with retrieved state-action-result exemplars that are generated in real time. When evaluated on the REAL web browsing benchmark, PRAXIS improves task completion accuracy, reliability, and cost efficiency across different foundation model backbones, and shows preliminary generalization to unseen tasks in similar environments. These results demonstrate that PRAXIS enables the practical adoption of AI agents in fast-evolving stateful environments by helping them learn new procedures effectively.

Paper Structure

This paper contains 19 sections, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. AI Agents and Procedural Learning
  3. Web Agents and the Browser Environment
  4. Related Work
  5. External Memory for AI Chatbots
  6. Experience‑Based Self‑Improvement and Workflow Memories
  7. Methods
  8. Altrina Agent
  9. State-Dependent Memory
  10. Benchmark
  11. Results
  12. Procedural memory improves agent accuracy
  13. Procedural memory improves agent reliability
  14. Procedural memory improves agent efficiency
  15. Procedural memory performance scales with retrieval breadth
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Performance on REAL benchmark for different models with (blue) and without (grey) procedural memory.
  • Figure 2: Agent reliability on REAL benchmark for different models with (blue) and without (grey) procedural memory.
  • Figure 3: Performance as a function of retrieval breadth.