Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approach

TL;DR

This paper interrogates whether object-centric deep neural networks can achieve abstract visual reasoning by testing them on a suite of tasks derived from comparative cognition (MTS, SD, SOSD, RMTS) and assessing out-of-distribution generalization. It compares a ResNet-50 baseline with six object-centric models, including ViT-based and Slot Attention–based architectures, across five simulations with varying perceptual features and training regimes. The results show that object-centric models can improve out-of-distribution generalization on some simpler tasks (notably MTS and SD) but fail on more demanding relational tasks (SOSD and RMTS), and that performance is highly task- and dataset-dependent; even the strongest models do not demonstrate robust, abstract relational reasoning. Attribution analyses indicate these models often rely on object regions rather than global scene structure, underscoring that current object-centric approaches are not sufficient for human-like relational generalization. The findings emphasize the need for architectures that separately represent objects and their relations and for more severe testing to avoid overclaiming relational capabilities in neural systems, guiding future work toward integrating binding mechanisms and configural representations.

Abstract

Achieving visual reasoning is a long-term goal of artificial intelligence. In the last decade, several studies have applied deep neural networks (DNNs) to the task of learning visual relations from images, with modest results in terms of generalization of the relations learned. However, in recent years, object-centric representation learning has been put forward as a way to achieve visual reasoning within the deep learning framework. Object-centric models attempt to model input scenes as compositions of objects and relations between them. To this end, these models use several kinds of attention mechanisms to segregate the individual objects in a scene from the background and from other objects. In this work we tested relation learning and generalization in several object-centric models, as well as a ResNet-50 baseline. In contrast to previous research, which has focused heavily in the same-different task in order to asses relational reasoning in DNNs, we use a set of tasks -- with varying degrees of difficulty -- derived from the comparative cognition literature. Our results show that object-centric models are able to segregate the different objects in a scene, even in many out-of-distribution cases. In our simpler tasks, this improves their capacity to learn and generalize visual relations in comparison to the ResNet-50 baseline. However, object-centric models still struggle in our more difficult tasks and conditions. We conclude that abstract visual reasoning remains an open challenge for DNNs, including object-centric models.

Figures (14)

• Figure 1: Visual reasoning tasks. MTS: match-to-sample; SD: same-different; SOSD: second-order same-different; RMTS: relational match-to-sample. See text for details.
• Figure 2: Models tested.
• Figure 3: Positive and negative MTS examples per dataset.
• Figure 4: Accuracy by task, model and dataset. Error bars are standard errors of the mean.
• Figure 5: Model attributions by task, model and condition. For each task the first row shows the attributions of a simulated model with perfect object segregation. Model attributions were calculated through the integrated gradients method and a logarithmic scale clipped at $1e-4$ was used for the visualization.