Inference of Abstraction for a Unified Account of Reasoning and Learning

TL;DR

This work addresses the challenge of unifying reasoning and learning under uncertainty by proposing a Bayesian-inspired probabilistic framework in which data $D$ generate symbolic knowledge in a propositional language $L$ via abstraction. The core is a generative reasoning model $p(L,M,D; μ)$ that maps data to world-models $M$ and then to the truth values of formulas, with $p(α|M=m_n)$ governed by a parameter $μ$ to control abstraction. The approach recovers classical entailment at $μ=1$ and yields empirical, all-nearest-neighbour–like reasoning as $μ$ approaches 1, while intermediate $μ$ values yield smoothed, weighted inferences. Empirically, the framework generalizes and improves upon a $k$-nearest-neighbour baseline on MNIST for digit prediction (and related image-generation tasks), illustrating a principled bridge between logic and machine learning and a path toward neuro-symbolic AI.

Abstract

Inspired by Bayesian approaches to brain function in neuroscience, we give a simple theory of probabilistic inference for a unified account of reasoning and learning. We simply model how data cause symbolic knowledge in terms of its satisfiability in formal logic. The underlying idea is that reasoning is a process of deriving symbolic knowledge from data via abstraction, i.e., selective ignorance. The logical consequence relation is discussed for its proof-based theoretical correctness. The MNIST dataset is discussed for its experiment-based empirical correctness.

Figures (7)

• Figure 1: A schematic of how the probability distribution over data determines the probability distribution over logical formulas. For simplicity, an arrow is omitted if the formula at the end of the arrow is false in the model at the start of the arrow and if the model at the end of the arrow is not supported by the data at the start of the arrow.
• Figure 2: An illustration of the assumptions of $[\![\Delta]\!]=[\![\![\Delta]\!]\!]$ and $[\![\Delta]\!]\neq\emptyset$ for reasoning of $\alpha\in L$ from $\Delta\subseteq L$ using the generative reasoning model $p(L,M,D;\mu=1)$. Each arrow from a datum to model, denoted respectively by a black circle on the top layer and a cell on the middle layer, represents that the datum supports the model. Each model with an incoming arrow thus has a non-zero probability. A model is coloured in green (resp. blue) if all the formulas in $\Delta$ are (resp. $\alpha$ is) true in the model.
• Figure 3: An illustration of the assumption of $[\![\![\Delta]\!]\!]\neq\emptyset$ for reasoning of $\alpha\in L$ from $\Delta\subseteq L$ using the generative reasoning model $p(L,M,D;\mu=1)$. The assumption of $[\![\Delta]\!]=[\![\![\Delta]\!]\!]$ assumed in Section \ref{['sec:consistency']} and illustrated in Figure \ref{['fig:consistent_reasoning']} is cancelled. It is shown that no data supports some of the models satisfying $\alpha$ and all the formulas in $\Delta$.
• Figure 4: Each cell of the grid is a model of $L$. The training and test images are shown above and below the grid, respectively. The blue cells on the top-left grid show that the prediction fails with $\mu=1$, since no training image is found in the models of the test image. The light blue cells on the top-right grid show that the prediction succeeds with $\mu\to 1$, since the limit expands the models of the test image until its best matched training image is found. The bottom left and right grids illustrate $\mu\in(0.5,1)$ and $\mu=0.5$, respectively.
• Figure 5: The prediction fails with $\mu\to 1$, since the test image and its nearest training image have different digits (see the medium blue cells). It can succeed with $\mu\in(0.5,1)$, since the models of the test image is expanded beyond its nearest training image for its second and further nearest training images (see the light blue cells). The curves on the right show the values of Expressions (\ref{['eq:prediction2_1']}), (\ref{['eq:prediction2_2']}) and (\ref{['eq:prediction2_3']}).

Theorems & Definitions (20)

• Proposition 1
• proof
• Example 1
• Definition 1: Possibility
• Theorem 1
• proof
• Corollary 1
• proof
• Example 2
• Theorem 2