Probabilistic Reasoning via Deep Learning: Neural Association Models

TL;DR

This paper introduces Neural Association Models (NAM) to address probabilistic reasoning by learning the conditional association $\Pr(E_2|E_1)$ between arbitrary events. Two NAM architectures are proposed: standard deep neural networks (DNN) and relation-modulated neural nets (RMNN), enabling end-to-end learning from data without explicit dependency graphs. Across tasks including Recognizing Textual Entailment, multi-relational KB triple classification, and CN14 commonsense reasoning, NAMs outperform conventional baselines, with RMNN offering superior knowledge transfer to unseen relations. The authors also extend NAMs to Winograd Schema challenges by automatically collecting cause-effect pairs and demonstrate modest yet meaningful gains, highlighting NAMs' potential for scalable, commonsense reasoning while acknowledging room for further progress.

Abstract

In this paper, we propose a new deep learning approach, called neural association model (NAM), for probabilistic reasoning in artificial intelligence. We propose to use neural networks to model association between any two events in a domain. Neural networks take one event as input and compute a conditional probability of the other event to model how likely these two events are to be associated. The actual meaning of the conditional probabilities varies between applications and depends on how the models are trained. In this work, as two case studies, we have investigated two NAM structures, namely deep neural networks (DNN) and relation-modulated neural nets (RMNN), on several probabilistic reasoning tasks in AI, including recognizing textual entailment, triple classification in multi-relational knowledge bases and commonsense reasoning. Experimental results on several popular datasets derived from WordNet, FreeBase and ConceptNet have all demonstrated that both DNNs and RMNNs perform equally well and they can significantly outperform the conventional methods available for these reasoning tasks. Moreover, compared with DNNs, RMNNs are superior in knowledge transfer, where a pre-trained model can be quickly extended to an unseen relation after observing only a few training samples. To further prove the effectiveness of the proposed models, in this work, we have applied NAMs to solving challenging Winograd Schema (WS) problems. Experiments conducted on a set of WS problems prove that the proposed models have the potential for commonsense reasoning.

Figures (10)

• Figure 1: Example of association between events.
• Figure 2: The NAM framework in general.
• Figure 3: The DNN structure for NAMs.
• Figure 4: The relation-modulated neural networks (RMNN).
• Figure 5: Accuracy of different relations in CN14.