Learning Interpretable Differentiable Logic Networks for Tabular Regression

TL;DR

The paper extends Differentiable Logic Networks (DLNs) from classification to tabular regression by introducing a SumLayer that turns binary rule activations into a continuous prediction, while preserving interpretability. It presents a unified end-to-end training framework that jointly learns thresholding, logic functions, and connections with differentiable relaxations and temperature annealing. Empirical results on 15 regression benchmarks show DLNs achieve competitive accuracy with strong interpretability and substantially lower inference cost than black-box baselines, occupying a favorable Pareto frontier for efficiency and performance. This work positions regression-DLN as a transparent, cost-effective alternative for regression tasks where model transparency and computational efficiency are important.

Abstract

Neural networks (NNs) achieve outstanding performance in many domains; however, their decision processes are often opaque and their inference can be computationally expensive in resource-constrained environments. We recently proposed Differentiable Logic Networks (DLNs) to address these issues for tabular classification based on relaxing discrete logic into a differentiable form, thereby enabling gradient-based learning of networks built from binary logic operations. DLNs offer interpretable reasoning and substantially lower inference cost. We extend the DLN framework to supervised tabular regression. Specifically, we redesign the final output layer to support continuous targets and unify the original two-phase training procedure into a single differentiable stage. We evaluate the resulting model on 15 public regression benchmarks, comparing it with modern neural networks and classical regression baselines. Regression DLNs match or exceed baseline accuracy while preserving interpretability and fast inference. Our results show that DLNs are a viable, cost-effective alternative for regression tasks, especially where model transparency and computational efficiency are important.

Figures (9)

• Figure 1: A simplified regression DLN example. Continuous input features are first binarized by a ThresholdLayer, producing a binary vector. This vector is then processed by successive LogicLayers composed of two-input Boolean operators. The activations of the final LogicLayer are combined by a SumLayer that computes a weighted sum, yielding the real-valued prediction.
• Figure 2: Training workflow for a regression DLN.
• Figure 3: Illustration of the training process. The network learns neuron functions and connections simultaneously: key details are highlighted for clarity.
• Figure 4: Comparison of $R^{2}$ and the number of operations required for inference across models. We plot results for 32, 64, and 128 hyperparameter-search trials and draw the Pareto frontier for each case; DLN is Pareto-optimal in all three.
• Figure 5: Distribution of $R^{2}$ across datasets for each model, averaged over 10 random seeds and 128 hyperparameter-search trials.