MemER: Scaling Up Memory for Robot Control via Experience Retrieval

TL;DR

This work proposes a hierarchical policy framework, where the high-level policy is trained to select and track previous relevant keyframes from its experience and the high-level policy uses selected keyframes and the most recent frames when generating text instructions for a low-level policy to execute.

Abstract

Humans routinely rely on memory to perform tasks, yet most robot policies lack this capability; our goal is to endow robot policies with the same ability. Naively conditioning on long observation histories is computationally expensive and brittle under covariate shift, while indiscriminate subsampling of history leads to irrelevant or redundant information. We propose a hierarchical policy framework, where the high-level policy is trained to select and track previous relevant keyframes from its experience. The high-level policy uses selected keyframes and the most recent frames when generating text instructions for a low-level policy to execute. This design is compatible with existing vision-language-action (VLA) models and enables the system to efficiently reason over long-horizon dependencies. In our experiments, we finetune Qwen2.5-VL-7B-Instruct and $π_{0.5}$ as the high-level and low-level policies respectively, using demonstrations supplemented with minimal language annotations. Our approach, MemER, outperforms prior methods on three real-world long-horizon robotic manipulation tasks that require minutes of memory. Videos and code can be found at https://jen-pan.github.io/memer/.

Figures (9)

• Figure 1: MemER Overview. We present MemER, a framework for scaling up memory in robotic control. MemER is able to utilize task-relevant past information to effectively reason through three complex long-horizon tasks, using a single policy trained with only a small number of expert demonstrations.
• Figure 2: Architecture of MemER. The high-level policy processes task instructions, selected keyframes (if any), and recent images from base and wrist-mounted cameras to generate low-level language subtasks and candidate keyframes (if any). The low-level policy uses the subtask, current image, and robot joint states to produce actions. The candidate keyframe(s) are processed by the keyframe filter to obtain the selected keyframes for input during the next step of inference.
• Figure 3: 1D single-linkage over nominated frames. At each timestep, the high-level policy nominates candidate keyframe(s), as highlighted in orange. All candidate keyframes are aggregated across time with 1D single-linkage using a merge distance of $d=5$ frames, yielding disjoint clusters. For each cluster, the colored bars indicate nominations for the observation at that timestamp, with bar height proportional to the number of nominations received. We select one representative frame per cluster by taking the median keyframe of all the candidates, and add that frame to memory.
• Figure 4: Tasks used in our evaluations. Across three domains, we evaluate complex instructions, intermediate subtasks, and keyframe predictions. We report performance across 20 trials per task per method.
• Figure 5: Main Results. Our method clearly outperforms the no history, short history (8 frames of context), and long history (32 frames of context) baselines on the three long-horizon memory-based tasks. It is on par with the human high level policy.