Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining

TL;DR

This work proposes an automatic keyframe selector that learns a discriminative embedding space, effectively identifying distinct state transitions in Vision-Language-Action models, and designs a progress-aware query mechanism that dynamically retrieves historical frames based on their temporal relevance to the current execution phase.

Abstract

Existing Vision-Language-Action (VLA) models often struggle to generalize to long-horizon tasks due to their heavy reliance on immediate observations. While recent studies incorporate retrieval mechanisms or extend context windows to handle procedural tasks, they often struggle to capture Non-Markovian dependencies, where optimal actions rely solely on specific past states rather than the current observation. To address this, we introduce Keyframe-Chaining VLA, a framework that extracts and links key historical frames to model long-horizon dependencies. Specifically, we propose an automatic keyframe selector that learns a discriminative embedding space, effectively identifying distinct state transitions. To capture task-critical information, we design a progress-aware query mechanism that dynamically retrieves historical frames based on their temporal relevance to the current execution phase. These selected keyframes are integrated into the VLA as interleaved visual tokens, explicitly grounding the policy in the long-horizon temporal context. Finally, we introduce a suite of four Non-Markovian manipulation tasks built upon the ManiSkill simulator to measure task success rates. Experimental results demonstrate that our method achieves superior performance, effectively tackling robot manipulation tasks characterized by long-horizon temporal dependencies. Code is available at https://github.com/cytoplastm/KC-VLA.

Figures (6)

• Figure 1: (a) Unlike standard VLAs openvlaoctodp that rely on dense sliding windows or context compression contextvlahamlet, our Keyframe-Chaining VLA explicitly selects sparse semantic keyframes, avoiding computational redundancy. (b) The other three paradigms in (a) suffer from limited temporal receptive fields, failing to capture distant critical cues. In contrast, our KSM-based retrieval achieves a global temporal receptive field, enabling the agent to access long-range historical information and effectively resolve state aliasing. (c) Our framework achieves state-of-the-art performance, significantly outperforming baselines in both ManiSkill memory-dependent tasks and real-world long-horizon deployment.
• Figure 2: Overview of the Keyframe-Chaining VLA framework. The architecture operates via two synergistic modules. (Left) Keyframe Selection Module: Adopts a two-stage design where Stage I optimizes a visual encoder via unified metric learning, and Stage II employs a Task-Modulated Query network to compute phase-specific confidence scores $s_t$. (Top Right) Greedy Temporal Smoothing: A stabilizing mechanism designed to filter detection jitter. It greedily updates provisional candidates to capture the latest valid state and finalizes the keyframe only after passing a temporal validation window. (Bottom Right) VLA Module: The policy integrates the sparse semantic history of detected keyframes with the current observation to generate actions via Flow Matching.
• Figure 3: Overview of the four custom Non-Markovian ManiSkill tasks. Red bounding boxes indicate the ground-truth keyframes utilized for training. Please refer to the Appendix for the detailed definitions of keyframe events for each task.
• Figure 4: Real-world Experimental Setup.
• Figure 5: The structured system prompt used to condition the VLA policy. The {Language Instruction} placeholder is replaced by the specific natural language command for the current episode (e.g., "Stack the cubes based on their initial vertical order").