MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learning

TL;DR

MetaSlot addresses the fixed-slot limitation in object-centric learning by introducing a global vector-quantized codebook of object prototypes and a two-stage assembly with de-duplication. It aligns first-stage slot representations to prototypes, prunes duplicates, and then refines the selected prototypes via Masked Slot Attention with progressive noise to stabilize convergence. The method yields substantial gains in object discovery and set prediction across synthetic and real-world datasets, with clear interpretability of prototype-to-object bindings. This approach enables adaptive slot counts, improved object-level representations, and robust performance across diverse decoder architectures, marking a step toward flexible, scalable OCL systems.

Abstract

Learning object-level, structured representations is widely regarded as a key to better generalization in vision and underpins the design of next-generation Pre-trained Vision Models (PVMs). Mainstream Object-Centric Learning (OCL) methods adopt Slot Attention or its variants to iteratively aggregate objects' super-pixels into a fixed set of query feature vectors, termed slots. However, their reliance on a static slot count leads to an object being represented as multiple parts when the number of objects varies. We introduce MetaSlot, a plug-and-play Slot Attention variant that adapts to variable object counts. MetaSlot (i) maintains a codebook that holds prototypes of objects in a dataset by vector-quantizing the resulting slot representations; (ii) removes duplicate slots from the traditionally aggregated slots by quantizing them with the codebook; and (iii) injects progressively weaker noise into the Slot Attention iterations to accelerate and stabilize the aggregation. MetaSlot is a general Slot Attention variant that can be seamlessly integrated into existing OCL architectures. Across multiple public datasets and tasks--including object discovery and recognition--models equipped with MetaSlot achieve significant performance gains and markedly interpretable slot representations, compared with existing Slot Attention variants.

Figures (6)

• Figure 1: Overview of the MetaSlot framework (depicted on the DINOSAUR backbone for clarity; agnostic to the underlying object-centric architecture). (i) We build and continually update a codebook of "prototype slots" by vector-quantizing slots sampled across the dataset. (ii) Input features $\bm Z$ are first aggregated via Slot Attention to produce an intermediate slot set $\bm S^{\mathrm{mid}}$; we then remove duplicate slots in $\bm S^{\mathrm{mid}}$ by matching them against the prototype slots, yielding the masked subset $\hat{\bm S}_{\mathrm{mask}}$. (iii) Finally, $\hat{\bm S}_{\mathrm{mask}}$ is passed through Masked Slot Attention with progressively attenuated noise to generate the refined slots $\bm S^{\mathrm{final}}$, which are then decoded to reconstruct the original input.
• Figure 2: Qualitative results show that MetaSlot’s dynamic slot allocation mitigates BO-QSA’s over-segmentation, such as splitting a train into unrelated parts, due to its fixed slot count.
• Figure 3: We visualize slot representations initialized from different prototype slots on the COCO dataset, where each column shows a specific initialization slot index--prototype slots in MetaSlot (out of 512) and fixed slots in BO-QSA BO-QSA. MetaSlot’s prototype-based initialization yields slots with strong semantic consistency and object binding (e.g., #slot 208 for trucks, #slot 445 for persons). By comparison, the fixed slots in BO-QSA frequently lack such coherent semantic grouping.
• Figure 4: Ablation study on architectural components. Backbone: DINOv2 ViT (s/14).
• Figure 5: Training curves for MetaSlot$_\mathrm{Mlp}$ with/without progressively attenuated noise.