Learning to Compose: Improving Object Centric Learning by Injecting Compositionality

TL;DR

This work targets object-centric learning by addressing the misalignment between the auto-encoding objective and learning compositional representations. It introduces a compositional objective that constructs composite slot representations from two images and evaluates them with a generative prior, effectively pushing slots to capture reusable, object-level components. The method combines an auto-encoding path with a composition path (using strategies like random sampling and shared slot initialization) and optimizes a total loss that includes prior, diffusion, reconstruction, and regularization terms. Across four datasets, the approach yields stronger unsupervised segmentation results and demonstrates robust performance against changes in the number of slots, encoder type, and decoder capacity, highlighting improved object-centric disentanglement and compositional generation. The findings suggest the proposed objective can enhance practical object-centric learning by promoting true compositionality and resilience to architectural biases.

Abstract

Learning compositional representation is a key aspect of object-centric learning as it enables flexible systematic generalization and supports complex visual reasoning. However, most of the existing approaches rely on auto-encoding objective, while the compositionality is implicitly imposed by the architectural or algorithmic bias in the encoder. This misalignment between auto-encoding objective and learning compositionality often results in failure of capturing meaningful object representations. In this study, we propose a novel objective that explicitly encourages compositionality of the representations. Built upon the existing object-centric learning framework (e.g., slot attention), our method incorporates additional constraints that an arbitrary mixture of object representations from two images should be valid by maximizing the likelihood of the composite data. We demonstrate that incorporating our objective to the existing framework consistently improves the objective-centric learning and enhances the robustness to the architectural choices.

Figures (12)

• Figure 1: Overview of our method. Our framework consists of two paths: an auto-encoding path and a composition path. The auto-encoding path ensures slot representations encode relevant information about an image. In contrast, the composition path encourages the compositionality of the representations by constructing the composite representation through the mixture of slots from two separate images (Section \ref{['subsec:slot_mixing']}), and assessing the quality of the composite image by the generative prior (Section \ref{['subsec:compsitional_objective']}). The encoder is jointly optimized by both paths.
• Figure 2: Comparison results on unsupervised object segmentation. We evaluate the how well the slot attention masks coincide with the ground-truth objects using FG-ARI, mIoU, and mBO (The higher is better). All results are evaluated on held-out validation set.
• Figure 3: Qualitative results on unsupervised object segmentation. The baselines tend to split an object into different slots (CleverTex) and/or combine different objects and background into a single (MultiShapeNet, PGR, Super-CLEVR). On the other hand, our method produces consistently better object masks, showing improved disentanglement of objects and background in all datasets. More results are presented in the Figure \ref{['fig:unsupervised_seg_appendix']}. Zoom in for better view.
• Figure 4: Robustness against various architectural biases. We evaluate the robustness of our model various different number of slots, encoder architectures, and decoder capacities. Results based on mIoU and mBO are presented in Figure \ref{['fig:full_ablation_appendix']}.
• Figure 5: Investigating object representation through compositional generation. We investigate the compositionality of learned representations by removing (red arrow) and adding (blue arrow) object slots between two images and generating the composite image. More results are in Figure \ref{['fig:Composition_two_imgs_appendix']}.