Efficient Object-centric Representation Learning with Pre-trained Geometric Prior

TL;DR

This work tackles object-centric representation learning in complex multi-object videos under weak supervision by bootstrapping from geometry-rich pre-trained visual encoders. It introduces an Attentional Slot Decoder that enables rich cross-slot interactions with a lightweight visual decoder, trained via a feature-space reconstruction objective using a CroCo-based backbone. The approach demonstrates strong object discovery and segmentation, surpassing several baselines and achieving competitive performance with lower memory requirements, across MOVi datasets. The findings suggest that leveraging geometric priors from pre-trained models can significantly improve object-centric learning and broaden practical deployment in real-world scenarios.

Abstract

This paper addresses key challenges in object-centric representation learning of video. While existing approaches struggle with complex scenes, we propose a novel weakly-supervised framework that emphasises geometric understanding and leverages pre-trained vision models to enhance object discovery. Our method introduces an efficient slot decoder specifically designed for object-centric learning, enabling effective representation of multi-object scenes without requiring explicit depth information. Results on synthetic video benchmarks with increasing complexity in terms of objects and their movement, object occlusion and camera motion demonstrate that our approach achieves comparable performance to supervised methods while maintaining computational efficiency. This advances the field towards more practical applications in complex real-world scenarios.

Figures (4)

• Figure 1: Attention map of different pre-trained self-supervised vision models on a video from the Movi-C dataset. While DINO and MSN show localised attention towards foreground objects, MAE and CroCo exhibit a diffused, global attention map.
• Figure 2: Our overall architecture and training pipeline including the reconstruction objective on the pre-trained geometric features and using an Attentional Slot Decoder for efficient learning.
• Figure 3: Indicative qualitative results of our Attentional Decoder on the Movi-A dataset. In each row of images we visualise the input video, the RGB reconstruction target of the input (Rec.), the ground truth object masks (Mask) and the predicted object masks from our object representations (Pred.).
• Figure 4: Two examples of unsupervised object segmentation with our method on MOVi-E. The horizontal axis represents different timeframes in a clip while the vertical axis shows our prediction. The first row shows input images with ground truth masks overlaid, the second row is overlaid with segmentation from our slot attention encoder and the third row by our prediction from the attentional slot decoder.