Rethinking Intelligence: Brain-like Neuron Network

TL;DR

The work tackles the rigidity of conventional neural architectures by introducing Brain-like Neural Networks (BNN) and presenting LuminaNet, a self-evolving, convolution- and attention-free model. LuminaNet autonomously grows and rewires through four operations (splitting, growing, connecting, pruning) using a Two-Pass Forward mechanism, achieving competitive results on CIFAR-10 and text generation on TinyStories without traditional inductive biases. Key contributions include a formal BNN definition, the first self-architectural-evolving network, and extensive visualizations that illuminate the evolving topology and information flow, highlighting interpretability. The findings suggest that autonomous architectural evolution can yield strong performance and efficiency gains, offering a pathway toward deeper integration of AI with neuroscience-inspired principles.

Abstract

Since their inception, artificial neural networks have relied on manually designed architectures and inductive biases to better adapt to data and tasks. With the rise of deep learning and the expansion of parameter spaces, they have begun to exhibit brain-like functional behaviors. Nevertheless, artificial neural networks remain fundamentally different from biological neural systems in structural organization, learning mechanisms, and evolutionary pathways. From the perspective of neuroscience, we rethink the formation and evolution of intelligence and proposes a new neural network paradigm, Brain-like Neural Network (BNN). We further present the first instantiation of a BNN termed LuminaNet that operates without convolutions or self-attention and is capable of autonomously modifying its architecture. We conduct extensive experiments demonstrating that LuminaNet can achieve self-evolution through dynamic architectural changes. On the CIFAR-10, LuminaNet achieves top-1 accuracy improvements of 11.19%, 5.46% over LeNet-5 and AlexNet, respectively, outperforming MLP-Mixer, ResMLP, and DeiT-Tiny among MLP/ViT architectures. On the TinyStories text generation task, LuminaNet attains a perplexity of 8.4, comparable to a single-layer GPT-2-style Transformer, while reducing computational cost by approximately 25% and peak memory usage by nearly 50%. Code and interactive structures are available at https://github.com/aaroncomo/LuminaNet.

Figures (9)

• Figure 1: Visualization of the self-construced, self-evolved Brain-like Neuron Network architectures.
• Figure 2: Structure of the Neuron Cluster (left) and Two-Pass Forward method (right).
• Figure 3: Visualization of the neuron weight matrices in the Input Layer.
• Figure 4: MDS visualizations of the features extracted by Neuron Layer after removing the connections (top) and before removing the connections (bottom). R, G, and B correspond to the red, green, and blue channels, respectively; T/B indicate the top and bottom, and L/R indicate the left and right.
• Figure 5: Neuron-layer output activations. The x-axis denotes the neuron index and the y-axis denotes the activation magnitude. Forward indicates activations after Pass-1 (forward information only), whereas Backward indicates activations after Pass-2 (with feedback information).