Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progress

Abstract

Accurately estimating task progress is critical for embodied agents to plan and execute long-horizon, multi-step tasks. Despite promising advances, existing Vision-Language Models (VLMs) based methods primarily leverage their video understanding capabilities, while neglecting their complex reasoning potential. Furthermore, processing long video trajectories with VLMs is computationally prohibitive for real-world deployment. To address these challenges, we propose the Recurrent Reasoning Vision-Language Model ($\text{R}^2$VLM). Our model features a recurrent reasoning framework that processes local video snippets iteratively, maintaining a global context through an evolving Chain of Thought (CoT). This CoT explicitly records task decomposition, key steps, and their completion status, enabling the model to reason about complex temporal dependencies. This design avoids the high cost of processing long videos while preserving essential reasoning capabilities. We train $\text{R}^2$VLM on large-scale, automatically generated datasets from ALFRED and Ego4D. Extensive experiments on progress estimation and downstream applications, including progress-enhanced policy learning, reward modeling for reinforcement learning, and proactive assistance, demonstrate that $\text{R}^2$VLM achieves strong performance and generalization, achieving a new state-of-the-art in long-horizon task progress estimation. The models and benchmarks are publicly available at \href{https://huggingface.co/collections/zhangyuelin/r2vlm}{huggingface}.

Figures (8)

• Figure 1: $\text{R}^2$VLM performs multi-turn recurrent reasoning. It takes a video snippet and the CoT from the previous iteration as input, and outputs the updated CoT and a progress estimation.
• Figure 2: Framework of $\text{R}^2$VLM. It performs recurrent reasoning over the newly observed video snippet, while incorporating the historical CoT as context. The CoT records the completed, pending, and incomplete steps from earlier video snippets and is dynamically updated throughout the recurrent reasoning process. The model estimates task progress based on the proportion of completed steps, and supports diverse applications including progress estimation, reward modeling, and proactive assistance.
• Figure 3: An illustration of the CoT structure and its update process in recurrent reasoning.
• Figure 4: Comparison of MAE and inference time between full and incremental video inputs at different video durations.
• Figure 5: The comparison of the policy model's performance on a three-step task before and after online reinforcement learning. The rewards in the reinforcement learning process are provided by $\text{R}^2$VLM.