Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices

TL;DR

This survey analyzes how deep learning methods tackle Abstract Visual Reasoning through Raven's Progressive Matrices (RPMs). It surveys RPM benchmarks (PGM, RAVEN, I-RAVEN, RAVEN-FAIR), learning paradigms (supervised, auxiliary, contrastive, unsupervised) and model families (relational reasoning networks, hierarchical networks), highlighting generalisation challenges beyond the Neutral regime. Key findings show that while several models approach or exceed human performance on constrained benchmarks, robust out-of-distribution generalisation remains difficult, and smaller, well-designed architectures can outperform larger backbones. The work emphasizes the need for few-shot and cross-problem transfer capabilities, and for unbiased, diverse benchmarks to meaningfully assess machine intelligence in abstract reasoning tasks.

Abstract

Abstract visual reasoning (AVR) domain encompasses problems solving which requires the ability to reason about relations among entities present in a given scene. While humans, generally, solve AVR tasks in a "natural" way, even without prior experience, this type of problems has proven difficult for current machine learning systems. The paper summarises recent progress in applying deep learning methods to solving AVR problems, as a proxy for studying machine intelligence. We focus on the most common type of AVR tasks -- the Raven's Progressive Matrices (RPMs) -- and provide a comprehensive review of the learning methods and deep neural models applied to solve RPMs, as well as, the RPM benchmark sets. Performance analysis of the state-of-the-art approaches to solving RPMs leads to formulation of certain insights and remarks on the current and future trends in this area. We conclude the paper by demonstrating how real-world problems can benefit from the discoveries of RPM studies.

Figures (11)

• Figure 1: RPM example. Remarkably, humans are able to intuitively solve the challenge even without the exact definition of the task (that is presented in Section \ref{['sec:rpms']}). The matrix is governed by multiple abstract patterns. Each row contains circle slices of 3 different colours split among columns, whereas squares have constant colour in each row. Moreover, positions of objects in the third column are determined by logical XOR applied row-wise in the case of squares and logical OR in the case of circle slices. The correct answer is F.
• Figure 2: RPM taxonomy. A list of RPM benchmarks, learning methods and DL models considered in this paper.
• Figure 3: RPM notation. A sample RPM from the RAVEN dataset zhang2019raven, which consists of two parts -- the context and the answer panels. The goal is to complete the context with appropriate answer panel. The chosen answer panel must fulfil all abstract rules governing the content of the context panels. In the example there are three such rules applied: (1) the same set of 3 shapes is present in each row, (2) the shapes in a given row are of the same size, and (3) the shape color intensity in the third column equals the sum of color intensities in two previous columns. Hence, the correct answer is C.
• Figure 4: RPM examples. Correct answers are marked with a green boundary. Matrices from RAVEN, I-RAVEN and RAVEN-FAIR differ only in the way of generating answers, hence only a selected matrix from the I-RAVEN dataset is shown.
• Figure 5: RPM augmentation. The first row presents single rows from two I-RAVEN matrices with configurations 2x2Grid (left) and 3x3Grid (right). Rows 2--4 demonstrate image-level augmentations used in malkinski2020multilabel, whereas the last row illustrates structural perturbations applied in kim2020few.