Triadic Concept Analysis for Logic Interpretation of Simple Artificial Networks

TL;DR

The paper tackles the interpretability gap of artificial neural networks by deriving symbolic logic representations from a simple ReLU-based network trained on minterm values. It partitions the input space into partition cells using ReLU statuses, maps minterm weights to a three-dimensional bit tensor, and applies Formal Concept Analysis and Triadic Concept Analysis to extract exclusive triadic concepts that are translated into logic trees via a quantum-logic-inspired decision-tree framework. This yields a linear combination of logic trees whose evaluations approximate the original ANN scores while providing human-interpretable explanations of attribute interactions, as demonstrated on a transfusion dataset with 74% accuracy and a compact set of concepts. The work presents a principled bridge between subsymbolic and symbolic paradigms, enabling interpretable insight into how attribute interactions drive decisions, albeit with current limits to small input dimensions and future work extending to deeper or more complex networks.

Abstract

An artificial neural network (ANN) is a numerical method used to solve complex classification problems. Due to its high classification power, the ANN method often outperforms other classification methods in terms of accuracy. However, an ANN model lacks interpretability compared to methods that use the symbolic paradigm. Our idea is to derive a symbolic representation from a simple ANN model trained on minterm values of input objects. Based on ReLU nodes, the ANN model is partitioned into cells. We convert the ANN model into a cell-based, three-dimensional bit tensor. The theory of Formal Concept Analysis applied to the tensor yields concepts that are represented as logic trees, expressing interpretable attribute interactions. Their evaluations preserve the classification power of the initial ANN model.

Figures (9)

• Figure 1: Example logic trees: solid lines refer to non-negated attribute evaluations, while dashed lines refer to negated attribute evaluations. Arithmetic evaluations of the logic trees are as follows: $[a_1]^i=x_i[1]$ (left), $[\overline{a}_1 \land a_2]^i=(1-x_i[1]) \cdot x_i[2]$ (middle), and $[a_1 \lor (\overline{a}_1 \land a_2)]=x_i[1] + (1-x_i[1]) \cdot x_i[2]$ (right).
• Figure 2: Example lattice of partition cells identified by bit codes for the status of three ReLU nodes: the small number on the left indicates the number of zero-objects ($y_i=0$), and the small number on the right indicates the number of one-objects ($y_i=1$) in $TR$. Bold partitions are those that are of interest (non-empty).
• Figure 3: Dashed lines: lossy mapping of minterm weights to binary numbers ($\#bits=2$);
• Figure 4: Extracting exclusive triconcepts
• Figure 5: Example logic tree for $ex\_c[0]$; solid lines non-negated and dashed line means negated evaluation, paths to 0-leaves can be dropped; below the arithmetic evaluation for an object $x_i$ (discussed in Section \ref{['sec:interpretation']})