Interpretable Neural Networks with Random Constructive Algorithm

TL;DR

This work tackles the opaque parameterization of Random Weighted Neural Networks (RWNNs) by introducing Interpretable Neural Networks (INN) that embed spatial information to link hidden parameters with residuals. It proposes a geometric relationship and a node-pooling strategy to select parameters that promote convergence, and also presents a lightweight IN+ variant based on Greville updates for large-scale data. The key contributions are the interpretable spatial information constraint, the node-pooling mechanism, and two algorithmic implementations (INN and IN+) with demonstrated universal/infinite approximation capability. Empirically, INN and especially IN+ achieve faster convergence, lower training RMSE, and more compact networks across diverse benchmarks and real-world tasks, underscoring the practical value of interpretability in randomized neural architectures.

Abstract

This paper introduces an Interpretable Neural Network (INN) incorporating spatial information to tackle the opaque parameterization process of random weighted neural networks. The INN leverages spatial information to elucidate the connection between parameters and network residuals. Furthermore, it devises a geometric relationship strategy using a pool of candidate nodes and established relationships to select node parameters conducive to network convergence. Additionally, a lightweight version of INN tailored for large-scale data modeling tasks is proposed. The paper also showcases the infinite approximation property of INN. Experimental findings on various benchmark datasets and real-world industrial cases demonstrate INN's superiority over other neural networks of the same type in terms of modeling speed, accuracy, and network structure.

Figures (9)

• Figure 1: Network structure of INN.
• Figure 2: construction process of INN.
• Figure 3: Convergence of RMSE.
• Figure 4: KDF for four algorithms.
• Figure 5: Impact of $\lambda$ on KDF.