Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StreamingClaw Technical Report

Jiawei Chen, Zhe Chen, Chaoqun Du, Maokui He, Wei He, Hengtao Li, Qizhen Li, Zide Liu, Hao Ma, Xuhao Pan, Chang Ren, Xudong Rao, Xintian Shen, Chenfeng Wang, Tao Wei, Chengjun Yu, Pengfei Yu, Shengyu Yao, Chunpeng Zhou, Kun Zhan, Lihao Zheng, Pan Zhou, Xuhan Zhu, Yufei Zheng

Abstract

Emerging applications such as embodied intelligence, AI hardware, autonomous driving, and intelligent cockpits rely on a real-time perception-decision-action closed loop, posing stringent challenges for streaming video understanding. However, current agents mostly suffer from fragmented capabilities, such as supporting only offline video understanding, lacking long-term multimodal memory mechanisms, or struggling to achieve real-time reasoning and proactive interaction under streaming input. These shortcomings have become a key bottleneck for preventing agents from sustaining perception, making real-time decisions, and executing closed-loop actions in complex real-world environments, constraining their deployment and potential in dynamic, open physical worlds. To alleviate these issues, we propose StreamingClaw, a unified agent framework for streaming video understanding and embodied intelligence. Beyond maintaining full compatibility with the OpenClaw framework, it natively supports real-time, multimodal streaming interactions. StreamingClaw integrates five core capabilities: (1) It supports real-time streaming reasoning. (2) It supports reasoning about future events and proactive interaction under the online evolution of interaction objectives. (3) It supports multimodal long-term memory storage, hierarchical memory evolution, efficient memory retrieval, and memory sharing across multiple agents. (4) It supports a closed loop of perception-decision-action. In addition to conventional tools and skills, it also provides streaming tools and action-centric skills tailored for real-world physical environments. (5) It is compatible with the OpenClaw framework, allowing it to leverage the resources and support of the open-source community.

StreamingClaw Technical Report

Abstract

Emerging applications such as embodied intelligence, AI hardware, autonomous driving, and intelligent cockpits rely on a real-time perception-decision-action closed loop, posing stringent challenges for streaming video understanding. However, current agents mostly suffer from fragmented capabilities, such as supporting only offline video understanding, lacking long-term multimodal memory mechanisms, or struggling to achieve real-time reasoning and proactive interaction under streaming input. These shortcomings have become a key bottleneck for preventing agents from sustaining perception, making real-time decisions, and executing closed-loop actions in complex real-world environments, constraining their deployment and potential in dynamic, open physical worlds. To alleviate these issues, we propose StreamingClaw, a unified agent framework for streaming video understanding and embodied intelligence. Beyond maintaining full compatibility with the OpenClaw framework, it natively supports real-time, multimodal streaming interactions. StreamingClaw integrates five core capabilities: (1) It supports real-time streaming reasoning. (2) It supports reasoning about future events and proactive interaction under the online evolution of interaction objectives. (3) It supports multimodal long-term memory storage, hierarchical memory evolution, efficient memory retrieval, and memory sharing across multiple agents. (4) It supports a closed loop of perception-decision-action. In addition to conventional tools and skills, it also provides streaming tools and action-centric skills tailored for real-world physical environments. (5) It is compatible with the OpenClaw framework, allowing it to leverage the resources and support of the open-source community.
Paper Structure (23 sections, 8 figures)

This paper contains 23 sections, 8 figures.

Table of Contents

  1. Introduction
  2. Framework of StreamingClaw
  3. Multi-end Input Adaptation
  4. Pipeline of StreamingClaw
  5. StreamingReasoning Agent
  6. Streaming Inference
  7. Self-Planning Scheduling
  8. StreamingMemory Agent
  9. Multimodal Memory Storage
  10. Hierarchical Memory Evolution
  11. Efficient Memory Retrieval
  12. Cross-Agent Unified Memory
  13. StreamingProactivity Agent
  14. Training-free Adaption
  15. Training Adaption
  16. ...and 8 more sections

Figures (8)

  • Figure 1: Pipeline of StreamingClaw: Multi-end inputs and the user query are fed into the main–sub agents for perception and decision-making. The instructions produced by the main–sub agents then guide the execution of tools and skills, whose results are fed back to the main–sub agents, forming a closed-loop pipeline of perception–decision–action.
  • Figure 2: Flowchart of SteamingMind’s streaming inference and self-planning scheduling.
  • Figure 3: Flowchart of StreamingMemory’s memory storage, evolution, and retrieval algorithm process.
  • Figure 4: Interaction scenarios of the StreamingProactivity agent: time-aware and event-grounding.
  • Figure 5: Implementation of the StreamingProactivity agent: training-free adaptation and training-based adaptation.
  • ...and 3 more figures