Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AgentOCR: Reimagining Agent History via Optical Self-Compression

Lang Feng, Fuchao Yang, Feng Chen, Xin Cheng, Haiyang Xu, Zhenglin Wan, Ming Yan, Bo An

TL;DR

AgentOCR reframes long-horizon agent histories by representing accumulated observations and actions as compact visual memories, enabling a segment-based optical cache and a compression-aware RL objective. The method dramatically reduces token usage while preserving near-parity performance with text-based baselines on ALFWorld and search-based QA, and delivers substantial rendering speedups. This work demonstrates that visual histories can effectively substitute text-heavy logs for multi-turn reasoning, suggesting a practical path toward scalable, memory-efficient vision-language agents. The findings highlight the potential of visual compression and caching to mitigate token and compute bottlenecks in real-world agent deployment.

Abstract

Recent advances in large language models (LLMs) enable agentic systems trained with reinforcement learning (RL) over multi-turn interaction trajectories, but practical deployment is bottlenecked by rapidly growing textual histories that inflate token budgets and memory usage. We introduce AgentOCR, a framework that exploits the superior information density of visual tokens by representing the accumulated observation-action history as a compact rendered image. To make multi-turn rollouts scalable, AgentOCR proposes segment optical caching. By decomposing history into hashable segments and maintaining a visual cache, this mechanism eliminates redundant re-rendering. Beyond fixed rendering, AgentOCR introduces agentic self-compression, where the agent actively emits a compression rate and is trained with compression-aware reward to adaptively balance task success and token efficiency. We conduct extensive experiments on challenging agentic benchmarks, ALFWorld and search-based QA. Remarkably, results demonstrate that AgentOCR preserves over 95\% of text-based agent performance while substantially reducing token consumption (>50\%), yielding consistent token and memory efficiency. Our further analysis validates a 20x rendering speedup from segment optical caching and the effective strategic balancing of self-compression.

AgentOCR: Reimagining Agent History via Optical Self-Compression

TL;DR

AgentOCR reframes long-horizon agent histories by representing accumulated observations and actions as compact visual memories, enabling a segment-based optical cache and a compression-aware RL objective. The method dramatically reduces token usage while preserving near-parity performance with text-based baselines on ALFWorld and search-based QA, and delivers substantial rendering speedups. This work demonstrates that visual histories can effectively substitute text-heavy logs for multi-turn reasoning, suggesting a practical path toward scalable, memory-efficient vision-language agents. The findings highlight the potential of visual compression and caching to mitigate token and compute bottlenecks in real-world agent deployment.

Abstract

Recent advances in large language models (LLMs) enable agentic systems trained with reinforcement learning (RL) over multi-turn interaction trajectories, but practical deployment is bottlenecked by rapidly growing textual histories that inflate token budgets and memory usage. We introduce AgentOCR, a framework that exploits the superior information density of visual tokens by representing the accumulated observation-action history as a compact rendered image. To make multi-turn rollouts scalable, AgentOCR proposes segment optical caching. By decomposing history into hashable segments and maintaining a visual cache, this mechanism eliminates redundant re-rendering. Beyond fixed rendering, AgentOCR introduces agentic self-compression, where the agent actively emits a compression rate and is trained with compression-aware reward to adaptively balance task success and token efficiency. We conduct extensive experiments on challenging agentic benchmarks, ALFWorld and search-based QA. Remarkably, results demonstrate that AgentOCR preserves over 95\% of text-based agent performance while substantially reducing token consumption (>50\%), yielding consistent token and memory efficiency. Our further analysis validates a 20x rendering speedup from segment optical caching and the effective strategic balancing of self-compression.
Paper Structure (43 sections, 8 equations, 9 figures, 5 tables, 2 algorithms)

This paper contains 43 sections, 8 equations, 9 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement Learning for LLM Agents
  4. Optical Character Recognition
  5. Agent Memory
  6. Preliminaries
  7. Problem Setup
  8. Agentic Reinforcement Learning
  9. AgentOCR
  10. Optical Memory Encoding
  11. Memory buffer.
  12. Memory rendering.
  13. Segment Optical Caching
  14. Segment representation.
  15. Segment cache.
  16. ...and 28 more sections

Figures (9)

  • Figure 1: Comparison of text agent and AgentOCR. (a) Text agent accumulates a heavy token burden from raw text history. (b) Our AgentOCR requires significantly fewer visual tokens via optical self-compression.
  • Figure 2: Overview of AgentOCR. (a) Segment optical caching decomposes the history context into segments, reuses cached renderings via content keys, and assembles the optical memory by stacking segment images. (b) The agent receives the optical observation and history, selects an environment action, and a compression rate. (c) The agent is trained with RL, jointly optimizing task performance and token efficiency.
  • Figure 3: Vision-text compression efficiency. The bars (left axis) denote the success rate relative to the text-based agent baseline, while the lines (right axis) indicate the percentage of tokens saved.
  • Figure 4: The prompt template of text agent on ALFWorld.
  • Figure 5: The prompt template of AgentOCR on ALFWorld.
  • ...and 4 more figures