Learning Object-Centric Spatial Reasoning for Sequential Manipulation in Cluttered Environments

TL;DR

This work argues for a paradigm of specialized, decoupled systems and presents Unveiler, a framework that explicitly separates high-level spatial reasoning from low-level action execution, and demonstrates that the SRE's spatial reasoning transfers zero-shot to real scenes, and validate the full system on a physical robot requiring only geometric workspace calibration.

Abstract

Robotic manipulation in cluttered environments presents a critical challenge for automation. Recent large-scale, end-to-end models demonstrate impressive capabilities but often lack the data efficiency and modularity required for retrieving objects in dense clutter. In this work, we argue for a paradigm of specialized, decoupled systems and present Unveiler, a framework that explicitly separates high-level spatial reasoning from low-level action execution. Unveiler's core is a lightweight, transformer-based Spatial Relationship Encoder (SRE) that sequentially identifies the most critical obstacle for removal. This discrete decision is then passed to a rotation-invariant Action Decoder for execution. We demonstrate that this decoupled architecture is not only more computationally efficient in terms of parameter count and inference time, but also significantly outperforms both classic end-to-end policies and modern, large-model-based baselines in retrieving targets from dense clutter. The SRE is trained in two stages: imitation learning from heuristic demonstrations provides sample-efficient initialization, after which PPO fine-tuning enables the policy to discover removal strategies that surpass the heuristic in dense clutter. Our results, achieving up to 97.6\% success in partially occluded and 90.0\% in fully occluded scenarios in simulation, make a case for the power of specialized, object-centric reasoning in complex manipulation tasks. Additionally, we demonstrate that the SRE's spatial reasoning transfers zero-shot to real scenes, and validate the full system on a physical robot requiring only geometric workspace calibration; no learned components are retrained.

Figures (4)

• Figure 1: Unveiling occluded targets through strategic obstacle removal. Given a cluttered scene with objects of similar shapes and colors, our approach identifies which obstacles must be removed and in what order to access the target object (blue object), solving the spatial reasoning challenge before action execution.
• Figure 2: Unveiler system architecture. Independent training of the SRE and Action Decoder enables each component to specialize in its respective task while sharing a discrete object index at inference. The SRE processes scene and object-centric visual inputs through a transformer to select the optimal obstacle, and the Action Decoder generates rotation-invariant push-grasp parameters for the selected object.
• Figure 3: Example scenes illustrating occlusion levels of the target object. (Left) The target object is partially occluded, with some visible surface area. (Right) The target object is fully occluded, entirely covered by surrounding objects, and not directly visible in the input view.
• Figure 4: Scene overlay with selection probabilities. Target outlined in red, selected obstacle in green.