KCNet: An Insect-Inspired Single-Hidden-Layer Neural Network with Randomized Binary Weights for Prediction and Classification Tasks
Jinyung Hong, Theodore P. Pavlic
TL;DR
KCNet introduces a biologically inspired, single-hidden-layer neural network using sparse, randomized binary input-to-hidden connections and analytically learned output weights via ridge regression. A Dynamic Optimization Algorithm (DOA) learns which input connections to keep, guided by straight-through gradient estimates, and an Ensemble DOA strategy scales this approach by assembling multiple small DOA-adjusted matrices. The model achieves competitive performance on odor perception tasks—outperforming some data-driven baselines—and demonstrates reasonable results on MNIST, Fashion-MNIST, and EMNIST-Balanced without data augmentation or convolution, while offering fast training times. This work demonstrates that brain-inspired architectural motifs combined with tractable linear learning and input-selection strategies can yield efficient, scalable classifiers with practical impact for resource-constrained applications.
Abstract
Fruit flies are established model systems for studying olfactory learning as they will readily learn to associate odors with both electric shock or sugar rewards. The mechanisms of the insect brain apparently responsible for odor learning form a relatively shallow neuronal architecture. Olfactory inputs are received by the antennal lobe (AL) of the brain, which produces an encoding of each odor mixture across ~50 sub-units known as glomeruli. Each of these glomeruli then projects its component of this feature vector to several of ~2000 so-called Kenyon Cells (KCs) in a region of the brain known as the mushroom body (MB). Fly responses to odors are generated by small downstream neutrophils that decode the higher-order representation from the MB. Research has shown that there is no recognizable pattern in the glomeruli--KC connections (and thus the particular higher-order representations); they are akin to fingerprints--even isogenic flies have different projections. Leveraging insights from this architecture, we propose KCNet, a single-hidden-layer neural network that contains sparse, randomized, binary weights between the input layer and the hidden layer and analytically learned weights between the hidden layer and the output layer. Furthermore, we also propose a dynamic optimization algorithm that enables the KCNet to increase performance beyond its structural limits by searching for a more efficient set of inputs. For odorant-perception tasks that predict the perceptual properties of an odorant, we show that KCNet outperforms existing data-driven approaches, such as XGBoost. For image classification tasks, KCNet achieves reasonable performance on benchmark datasets (MNIST, Fashion-MNIST, and EMNIST) without any data-augmentation methods or convolutional layers and shows a particularly fast running time.