Systematic Abductive Reasoning via Diverse Relation Representations in Vector-symbolic Architecture

TL;DR

This work tackles RPM-style abstract visual reasoning by introducing Rel-SAR, a neuro-symbolic model that unifies perception and reasoning in a vector-symbolic framework. It builds diverse attribute representations using atomic HD vectors (RVs, NVs, CVs, BVs) and SHDRs for image panels and grids, connected through numerical and logical relation functions with inverse operations for rule execution. End-to-end and auxiliary-label training demonstrate strong performance on RAVEN and I-RAVEN, especially for position-based rules, while offering interpretable abductive reasoning via explicit relation functions. The approach advances interpretability, robustness, and systematic generalization in RPM reasoning, with potential impact on broader neuro-symbolic AI tasks that require compositional, rule-governed reasoning over perceptual inputs.

Abstract

In abstract visual reasoning, monolithic deep learning models suffer from limited interpretability and generalization, while existing neuro-symbolic approaches fall short in capturing the diversity and systematicity of attributes and relation representations. To address these challenges, we propose a Systematic Abductive Reasoning model with diverse relation representations (Rel-SAR) in Vector-symbolic Architecture (VSA) to solve Raven's Progressive Matrices (RPM). To derive attribute representations with symbolic reasoning potential, we introduce not only various types of atomic vectors that represent numeric, periodic and logical semantics, but also the structured high-dimentional representation (SHDR) for the overall Grid component. For systematic reasoning, we propose novel numerical and logical relation functions and perform rule abduction and execution in a unified framework that integrates these relation representations. Experimental results demonstrate that Rel-SAR achieves significant improvement on RPM tasks and exhibits robust out-of-distribution generalization. Rel-SAR leverages the synergy between HD attribute representations and symbolic reasoning to achieve systematic abductive reasoning with both interpretable and computable semantics.

Figures (5)

• Figure 1: Illustrations for RAVEN dataset.(a) An example of RPM test from RAVENzhang2019raven dataset. In an RPM test, there are 8 context panels and 8 candidate panels. Participants are required to identify the underlying rules governing various attributes within the context panels. Subsequently, participants use these rules to infer the attributes of the missing panel (represented by "?") and choose the most appropriate option (highlighted with a red box) from the answer panels. (b) The RAVEN dataset includes seven configurations: Center, 2x2Grid, 3x3Grid, Left-Right (L-R), Up-Down (U-D), Out-InCenter (O-IC) and Out-InGrid (O-IG) zhang2019raven. Four types of rules, i.e., Constant, Progression, Arithmetic, and Distribute Three, are applied to five attributes, i.e., Position, Number, Type, Size, and Color, in a row-wise manner. The I-RAVEN dataset hu2021stratified is a variant of RAVEN, where answer sets are generated using an attribute bisection tree.
• Figure 2: Atomic HD representations and relation functions.(a-d) The Rel-SAR model utilizes four types of atomic HD vectors. Random Vectors (RVs), sampled independently, are used to represent distinct and unrelated symbols or concepts. Numeric Vectors (NVs) are used to represent real numbers and support VSA-based addition-type arithmetic operations. Circular Vectors (CVs) represent periodic values and enable addition-type arithmetic operations with periodicity. Boolean Vectors (BVs), representing logical values of False and True, support VSA operations for logical reasoning. (e) In the RAVEN dataset, for a given attribute, the HD attribute representations in a row of three image panels involve binary or ternary relations. (f) Relation functions describe the numerical or logical relations between multiple HD vector representations $\boldsymbol{v}_{1:N}$, where $N=2$ for binary and $N=3$ for ternary relations. These relations are governed by the operator powers $OP_{1:M}$ and the output $\boldsymbol{r}$. (g) For a given relation defined by $OP_{1:M}$ and $\boldsymbol{r}$, inverse relation functions infer the last HD vector representation $\boldsymbol{v}_N$ according to the first $N-1$ representations $\boldsymbol{v}_{1:N-1}$.
• Figure 3: Structured HD representations (SHDR) for the image panel and the nxn Grid.(a) SHDR for the image panel (Equation \ref{['SHDR']}). Taking the Out-InGrid configuration as an example, an image panel contains multiple objects, each with four entity attributes: type, size, color, and existence. Through the first layer of role-filler binding, these attributes are combined to form a SHDR for each object. Additionally, at the layout level, each object has a position attribute. By applying a second layer of role-filler binding, the SHDR for the entire image panel is constructed. (b) SHDR for the nxn Grid (Equation \ref{['SHDR_nxn']}). Taking the 3x3Grid configuration as an example, the position vectors for all objects are represented using circular vectors (CVs) with a period of $3 \times 3 = 9$. The SHDR $\mathcal{C}^{3\mathrm{x}3}$ for this 3x3Grid is obtained by performing role-filler binding between the corresponding position vectors $\boldsymbol{p}_j$ and existence vectors $\boldsymbol{v}_j$. Due to the periodic nature of the position vectors, as all objects shift positions cyclically, the SHDR undergoes a binding operation with the position vectors corresponding to the magnitude of the shift.
• Figure 4: The Systematic Abductive Reasoning model with diverse relation representations (Rel-SAR).(a) Overall architecture of Rel-SAR model. Our model consists of a visual perception frontend, which processes object attributes for $8$ context and $8$ candidate image panels in a RPM test, and a reasoning backend that performs symbolic arithmetic and logical reasoning. The perception frontend utilizes a neural network, $f_{\theta}$, to obtain the SHDR of each image panel, and then perceives attributes in the form of HD representations required by the downstream reasoning. In the reasoning backend, the rule abduction module extracts rules for each attribute representation using relation functions. The rule execution module then predicts the missing panel's attribute representations based on inverse relation functions. Finally, the answer selection module compares the predicted attributes of the missing panel with those in the candidate panels and selects the option with the highest similarity. (b) Given the predicted SHDR for each panel, the SHDR of all objects and their corresponding HD attribute representations can be obtained via representation decomposition. Subsequently, the estimated HD attribute representations are refined in two steps: querying the frontend codebook and applying attention based on the backend codebook. This process produces HD attribute representations suitable for backend reasoning, including attributes such as type, size, color, number, and position. (c) In the rule abduction module, the rule learners $f_{\phi }^{Num}$ and $f_{\varphi }^{Lgc}$ predict the operator powers $\widehat{OP}_{1:M}$ for numerical and logical relation functions based on attributes in the context panels. These predicted $\widehat{OP}_{1:M}$ ensure that all binary or ternary relation input pairs ($\mathbb{V}^{N}, N=2,3$) produce the same output $\boldsymbol{\hat{r}}$ when processed through their respective relation functions. Therefore, the rule defined by $\widehat{OP}_{1:M}$ and $\boldsymbol{\hat{r}}$ with the highest overall $\boldsymbol{\hat{r}}$ similarity, also viewed as unnormalized probability, is considered the underlying rule. The rule execution module then predicts the attributes of the missing panel using inverse relation functions with the estimated rules.
• Figure 5: Attention weights for existence on 2X2Grid. For the existence attribute, the presence weights ($W_{j}^{exist}\left( 1 \right)$ in Equation \ref{['query_exist']}) represent the probability of an object being present at the $j$th position. The sum of these weights provides an approximate indication of the total number of objects in a panel. (a) When $k_{train}=1$, the model consistently predicts the presence of only one object in the panel, regardless of the actual test scenario. (b) When $k_{train}=2$, the model learns to distinguish between panels with varying numbers of objects, effectively adapting its feature extraction process.