SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation

TL;DR

The paper introduces LIBERO+ and SlotVLA to advance object–relation-centric robotic manipulation. By replacing dense visual tokens with a compact mix of object-centric slots and relation tokens, and by enforcing task-aware filtering, the approach achieves substantial token efficiency while maintaining competitive performance. A two-stage training regime couples an object-centric encoder with a relation encoder, guided by an LLM-based action decoder, and is evaluated on LIBERO+ across multiple task subsets. The work demonstrates notable efficiency gains and interpretable representations, while highlighting remaining challenges in scaling to cluttered, long-horizon scenarios and in providing explicit relational grounding.

Abstract

Inspired by how humans reason over discrete objects and their relationships, we explore whether compact object-centric and object-relation representations can form a foundation for multitask robotic manipulation. Most existing robotic multitask models rely on dense embeddings that entangle both object and background cues, raising concerns about both efficiency and interpretability. In contrast, we study object-relation-centric representations as a pathway to more structured, efficient, and explainable visuomotor control. Our contributions are two-fold. First, we introduce LIBERO+, a fine-grained benchmark dataset designed to enable and evaluate object-relation reasoning in robotic manipulation. Unlike prior datasets, LIBERO+ provides object-centric annotations that enrich demonstrations with box- and mask-level labels as well as instance-level temporal tracking, supporting compact and interpretable visuomotor representations. Second, we propose SlotVLA, a slot-attention-based framework that captures both objects and their relations for action decoding. It uses a slot-based visual tokenizer to maintain consistent temporal object representations, a relation-centric decoder to produce task-relevant embeddings, and an LLM-driven module that translates these embeddings into executable actions. Experiments on LIBERO+ demonstrate that object-centric slot and object-relation slot representations drastically reduce the number of required visual tokens, while providing competitive generalization. Together, LIBERO+ and SlotVLA provide a compact, interpretable, and effective foundation for advancing object-relation-centric robotic manipulation.

Figures (5)

• Figure 1: Comparison of visuomotor tokenization strategies. (a) Dense tokenizers generate hundreds of tokens across the scene, leading to computationally costly representations. (b) Object-centric tokenizer yields $N_K$ tokens, each representing an object. (c) Our object–relation-centric tokenizer yields $N_S$ object tokens and $N_R$ relation tokens, producing structured and efficient representations. Plot (2) shows that our method achieves higher success rates with fewer tokens compared to baselines on LIBERO-Goal.
• Figure 2: Overview of the LIBERO+ dataset.
• Figure 3: Overall framework of our proposed model. Stage-1 trains the Task-aware Object-Centric Encoder with slot attention and task-aware filtering. Stage-2 freezes Stage-1 parameters and introduces the Relation-Centric Encoder, enabling relational reasoning for final action decoding.
• Figure 4: Slot decomposition result. Task query: "Put the bowl on the stove". Task-relevant slots correctly bind to objects, while irrelevant slots scatter.
• Figure 5: Trajectory demonstration in simulation from exocentric views. Task query: "Put the bowl on the stove".