Implicit State Estimation via Video Replanning

TL;DR

This work tackles planning under uncertainty in video-based decision-making by learning from past interactions without explicit state models. It introduces implicit state estimation (ISE) via video replanning, which online updates a latent environment embedding and rejects previously failed plans to adapt plans. A diffusion-based video planner generates multiple candidate plans conditioned on the embedding (e.g., $M=7$ future frames of $128\times128$ resolution) and a rejection module selects the most novel plan relative to past failures. On the Meta-World System Identification Benchmark and real-world door-opening data, the method reduces replanning failures and improves plan quality, demonstrating robust online adaptation without explicit parameter identification.

Abstract

Video-based representations have gained prominence in planning and decision-making due to their ability to encode rich spatiotemporal dynamics and geometric relationships. These representations enable flexible and generalizable solutions for complex tasks such as object manipulation and navigation. However, existing video planning frameworks often struggle to adapt to failures at interaction time due to their inability to reason about uncertainties in partially observed environments. To overcome these limitations, we introduce a novel framework that integrates interaction-time data into the planning process. Our approach updates model parameters online and filters out previously failed plans during generation. This enables implicit state estimation, allowing the system to adapt dynamically without explicitly modeling unknown state variables. We evaluate our framework through extensive experiments on a new simulated manipulation benchmark, demonstrating its ability to improve replanning performance and advance the field of video-based decision-making.

Figures (13)

• Figure 1: Implicit state estimation via video replanning. Previous video planning methods rely on the first frame (in red). We propose to incorporate past failures (in blue) to improve future plans.
• Figure 2: Framework overview.(a) Generate and select plans The Video Plan Generator first generates a set of candidate video plans conditioned on first-frame $f_0$ and state embedding $e$. The Rejection Module then selects the plan that is least similar to the plans in the Failed Plan Buffer $B_P$. (b) Execute plan and update buffers. The Action Module interacts with the environment by following the selected plan $v^{p*}$. If the interaction trial is not successful, we update the Failed Plan Buffer $B_P$ and the Failed Interaction Buffer $B_I$. (c) Update belief. The state embedding $e$ is updated using the Retrieval Module based on the Failed Interaction Buffer $B_I$. Then, we generate the next batch of video plans according to the updated belief.
• Figure 3: Video Plan Generator and Identification Module. We implement the Video Plan Generator and the Identification Module as two video generative models with shared parameters, switched with a binary trigger is_planning. The Video Plan Generator only learns from successful videos in the experience dataset $D$, while the Identification Module is trained with all videos in $D$, including the failures, enabling the state-embedding refining process.
• Figure 4: Qualitative results of adapted video plans. We show qualitative examples of adaptation trials for each task in the Meta-World Identification Benchmark to illustrate the effectiveness of the proposed method. The framework generates new plans based on previously failed interaction and video plans.
• Figure 5: Rejection and retrieval modules. Both the Rejection and Retrieval Modules improve the performance from AVDC (Aciton Module only) and combining them (ours) yields the best performance. In this figure, the numbers of replans are normalized aggregated among the five tasks.