No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks

TL;DR

This work introduces task-based neurons as a new design paradigm for artificial networks, inspired by biological neural diversity. It presents a two-step framework: vectorized symbolic regression to derive a simple, shared-formula neuronal model, followed by parameterization to create a trainable aggregation function within a task-based network. Empirical results across synthetic data, public tabular benchmarks, and real-world tasks show that networks built from task-based neurons outperform those with traditional linear or random-polynomial neurons and can achieve competitive or superior performance with fewer parameters. The findings suggest that embedding task priors at the neuron level can yield robust, scalable improvements and motivate further integration with task-based architectures and hardware acceleration.

Abstract

Biologically, the brain does not rely on a single type of neuron that universally functions in all aspects. Instead, it acts as a sophisticated designer of task-based neurons. In this study, we address the following question: since the human brain is a task-based neuron user, can the artificial network design go from the task-based architecture design to the task-based neuron design? Since methodologically there are no one-size-fits-all neurons, given the same structure, task-based neurons can enhance the feature representation ability relative to the existing universal neurons due to the intrinsic inductive bias for the task. Specifically, we propose a two-step framework for prototyping task-based neurons. First, symbolic regression is used to identify optimal formulas that fit input data by utilizing base functions such as logarithmic, trigonometric, and exponential functions. We introduce vectorized symbolic regression that stacks all variables in a vector and regularizes each input variable to perform the same computation, which can expedite the regression speed, facilitate parallel computation, and avoid overfitting. Second, we parameterize the acquired elementary formula to make parameters learnable, which serves as the aggregation function of the neuron. The activation functions such as ReLU and the sigmoidal functions remain the same because they have proven to be good. Empirically, experimental results on synthetic data, classic benchmarks, and real-world applications show that the proposed task-based neuron design is not only feasible but also delivers competitive performance over other state-of-the-art models.

Figures (4)

• Figure 1: Two steps to establish a task-based neuron. a) The symbolic regression constructs an elementary neuronal model, which stacks all variables in a vector and regularizes each input variable to perform the same computation. b) The acquired elementary formula is parameterized to be learnable, serving as the aggregation function of a neuron.
• Figure 2: A schematic diagram of crossover operation.
• Figure 3: A schematic diagram of mutation operation.
• Figure 4: The proposed vectorized symbolic regression.