A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons and Adaptable Structure

TL;DR

X-Net introduces learnable activation functions and an adaptive, tree-structured architecture to supersede fixed-activation MLPs. Through alternating backpropagation, it optimizes both weights and activations, enabling dynamic growth and pruning at the neuron level. Across regression, classification, and scientific-discovery tasks, X-Net matches or exceeds MLP performance while using a fraction of parameters and often producing interpretable, equation-like models. This approach offers a path toward compact, transparent neural systems with strong cross-domain applicability in science and engineering.

Abstract

Multilayer perception (MLP) has permeated various disciplinary domains, ranging from bioinformatics to financial analytics, where their application has become an indispensable facet of contemporary scientific research endeavors. However, MLP has obvious drawbacks. 1), The type of activation function is single and relatively fixed, which leads to poor `representation ability' of the network, and it is often to solve simple problems with complex networks; 2), the network structure is not adaptive, it is easy to cause network structure redundant or insufficient. In this work, we propose a novel neural network paradigm X-Net promising to replace MLPs. X-Net can dynamically learn activation functions individually based on derivative information during training to improve the network's representational ability for specific tasks. At the same time, X-Net can precisely adjust the network structure at the neuron level to accommodate tasks of varying complexity and reduce computational costs. We show that X-Net outperforms MLPs in terms of representational capability. X-Net can achieve comparable or even better performance than MLP with much smaller parameters on regression and classification tasks. Specifically, in terms of the number of parameters, X-Net is only 3% of MLP on average and only 1.1% under some tasks. We also demonstrate X-Net's ability to perform scientific discovery on data from various disciplines such as energy, environment, and aerospace, where X-Net is shown to help scientists discover new laws of mathematics or physics.

Figures (5)

• Figure 1: Figure illustrates the dynamic changes in the network structure of X-Net during the training process.
• Figure 2: Figure a and Figure b show the fitting results of Formula \ref{['medv1']} for Scaled sound pressure level; Figure c displays the correlation coefficient matrix for various variables of Airfoil-self-noise; Figure d and Figure e show the fitting results of Formula \ref{['medv2']} for Scaled sound pressure level. Figure f displays the fitting results of the Formula\ref{['medv2']} for global temperature changes; Figures g through n show the fitting results of X-Net on univariate benchmarks. Figures o through v display the prediction outcomes of X-Net on the multivariate benchmark, where '-TV' denotes true values and '-PRE' represents predicted values;
• Figure 3: Figure a illustrates the algorithmic flowchart of X-Net; From Figure b, we observe a performance discrepancy between X-Net and MLP in the same regression task, with X-Net having a significantly lower network complexity compared to MLP; Figure c describes the forward propagation process of our algorithm in detail;
• Figure 4: Figure a depicts the change in algorithmic efficiency before and after the use of ada-$\alpha$; Figure b presents the correlation matrix among different variables in the Boston housing price prediction; Figure c demonstrates the schematic representation of housing price predictions using the variable 'RM'. It can be deduced that the housing prices are directly proportional to the variable 'RM', which is consistent with the findings presented in Figure c; Figure d displays the results of predicting housing prices using the variable 'LSTAT'. Figure e showcases the schematic representation of predicting housing prices using both 'RM' and 'LSTAT'.
• Figure 5: Figure a and Figure b show the prediction results of Formula \ref{['equ8']} for solar power generation; Figure c and Figure d display the fitting results of Formula \ref{['equ9']} for wind power generation data; Figure e represents the correlation coefficient matrix of the variables used in the wind power generation data.