Logic interpretations of ANN partition cells

TL;DR

A new method of interpretation is constructed between a simple ANN and logic that can analyze and manipulate the semantics of an ANN using the powerful tool set of logic.

Abstract

Consider a binary classification problem solved using a feed-forward artificial neural network (ANN). Let the ANN be composed of a ReLU layer and several linear layers (convolution, sum-pooling, or fully connected). We assume the network was trained with high accuracy. Despite numerous suggested approaches, interpreting an artificial neural network remains challenging for humans. For a new method of interpretation, we construct a bridge between a simple ANN and logic. As a result, we can analyze and manipulate the semantics of an ANN using the powerful tool set of logic. To achieve this, we decompose the input space of the ANN into several network partition cells. Each network partition cell represents a linear combination that maps input values to a classifying output value. For interpreting the linear map of a partition cell using logic expressions, we suggest minterm values as the input of a simple ANN. We derive logic expressions representing interaction patterns for separating objects classified as 1 from those classified as 0. To facilitate an interpretation of logic expressions, we present them as binary logic trees.

Figures (5)

• Figure 1: Example of a simple ANN
• Figure 2: ReLU partition cells $p_{\_\_}$ based on hyperplanes and a class separation polyline in $[0,1]^2$
• Figure 3: Trend diagrams for evaluations of logic expressions of different bit code levels
• Figure 4: Example training data for a logic expression $mw^e_{p,bcl}$ expressed over minterms: dashed lines in resulting QLDT denote negated evaluation; leaves are class decisions; tree represents logic expression $e_{p,blc}={a}_2\lor (\overline{a}_2\land a_1)={a}_2\lor {a}_1$ and $[e_{p,bcl}]^{o_i}=m_2(o_i[2])+((1-m_2(o_i[2]))*m_1(o_i[1]))$
• Figure 5: Logic expressions of bit code level $bcl=1$ (left) and $bcl=2$ (right) as trees; dashed line for negated evaluation and solid line for non-negated evaluation; a leaf $B$ stands for an active minterm for the banknote authentication; inactive minterms are not shown; $v,s,c,e$ are short notations of the input object attributes