Representing Positional Information in Generative World Models for Object Manipulation

TL;DR

This paper tackles the challenge of representing positional information in generative world models for object manipulation. It introduces two strategies—Position-Conditioned Policy (PCP) and Latent-Conditioned Policy (LCP)—to inject explicit positional information and enable multimodal goal specification, including visual targets, within object-centric latent spaces. Through extensive offline evaluations across Reacher, Cube Move, Shelf Place, and Pick&Place tasks, PCP and especially LCP outperform baselines like Dreamer and standard FOCUS, demonstrating improved data efficiency and robustness in robotic manipulation. The findings highlight the importance of direct target conditioning and object-centric latent representations for multimodal goal specification, with implications for broader multimodal and tactile sensing in embodied agents.

Abstract

Object manipulation capabilities are essential skills that set apart embodied agents engaging with the world, especially in the realm of robotics. The ability to predict outcomes of interactions with objects is paramount in this setting. While model-based control methods have started to be employed for tackling manipulation tasks, they have faced challenges in accurately manipulating objects. As we analyze the causes of this limitation, we identify the cause of underperformance in the way current world models represent crucial positional information, especially about the target's goal specification for object positioning tasks. We introduce a general approach that empowers world model-based agents to effectively solve object-positioning tasks. We propose two declinations of this approach for generative world models: position-conditioned (PCP) and latent-conditioned (LCP) policy learning. In particular, LCP employs object-centric latent representations that explicitly capture object positional information for goal specification. This naturally leads to the emergence of multimodal capabilities, enabling the specification of goals through spatial coordinates or a visual goal. Our methods are rigorously evaluated across several manipulation environments, showing favorable performance compared to current model-based control approaches.

Figures (11)

• Figure 1: Object positioning task with coordinates goal.
• Figure 1: Average score for 100 goal points equally distributed over the workspace. For each task, the environments are shown, in order, on the left. We also show the goal points' workspace, delimited by an orange dotted line, and the reference frames indicated with arrows. Performance is averaged over 3 seeds, ± indicates the standard error.
• Figure 2: The world model compresses visual observations and state vector into a latent state representation. Crucially, the compressed representation serves as input to the policy for action selection. The world model can either be flat, encoding a single latent state, or object-centric, where the latent representation consists of distinct latent states for each object. (top) Goal information is provided through the input state vector. (bottom): Both single and object-centric representations can be paired to a target-conditioned policy.
• Figure 3: left: examples of virtual targets visualization.top-right: Dreamer's success rate and reconstruction performance over target and entity position (end-effector position for reacher and cube position for the cube move environment). bottom-right: Equivalent for the FOCUS object-centric model. The success rate for both environments is defined as the entity of interest being within 5cm from the given target at the termination of the episode. Reconstruction errors are computed as L2-norm.
• Figure 4: Dreamer virtual visual goal modulation experiments on the Reacher environment. Value prediction from the value network is shown to highlight the policy's awareness of the lack of information with respect to the target goal.