A Network of Biologically Inspired Rectified Spectral Units (ReSUs) Learns Hierarchical Features Without Error Backpropagation

TL;DR

This work tackles the gap between biological vision and deep learning by introducing Rectified Spectral Units (ReSUs), a biologically plausible, backpropagation-free architecture that learns hierarchical features through local, self-supervised learning. Each ReSU projects a recent input history onto a canonical direction learned via past–future CCA and then rectifies the projection, enabling principled temporal feature extraction. In a two-layer network trained on natural scene translations, Layer 1 develops Drosophila-like temporal filters with SNR adaptation, while Layer 2 becomes direction-selective with connectomics-like synaptic patterns, supporting a biologically grounded, self-supervised path to deep representations. The results suggest ReSUs can model sensory circuits and scale to deeper networks, offering an interpretable alternative to traditional backpropagation-based architectures.

Abstract

We introduce a biologically inspired, multilayer neural architecture composed of Rectified Spectral Units (ReSUs). Each ReSU projects a recent window of its input history onto a canonical direction obtained via canonical correlation analysis (CCA) of previously observed past-future input pairs, and then rectifies either its positive or negative component. By encoding canonical directions in synaptic weights and temporal filters, ReSUs implement a local, self-supervised algorithm for progressively constructing increasingly complex features. To evaluate both computational power and biological fidelity, we trained a two-layer ReSU network in a self-supervised regime on translating natural scenes. First-layer units, each driven by a single pixel, developed temporal filters resembling those of Drosophila post-photoreceptor neurons (L1/L2 and L3), including their empirically observed adaptation to signal-to-noise ratio (SNR). Second-layer units, which pooled spatially over the first layer, became direction-selective -- analogous to T4 motion-detecting cells -- with learned synaptic weight patterns approximating those derived from connectomic reconstructions. Together, these results suggest that ReSUs offer (i) a principled framework for modeling sensory circuits and (ii) a biologically grounded, backpropagation-free paradigm for constructing deep self-supervised neural networks.

Figures (5)

• Figure 1: Drosophila-inspired two-layer ReSU neural network trained in the self-supervised setting on translating natural images in 1D. Photoreceptors (PR) report the contrast levels of the corresponding pixels. Temporal filters of the first layer neurons (L1, L3) are computed by using CCA on past-future input sequences and act on PR outputs followed by rectification in L1. Second layer neurons (T4) pool information from 3 adjacent pixels. Past-future CCA of the outputs of the first layer learns a directionally selective spatio-temporal filter for T4 (black arrows: preferred directions).
• Figure 2: Past–future CCA as a computational primitive applied to different data streams. (A) A sensory stimulus is represented by past and future lag vectors. Each neuron computes its output as a linear projection of the input onto a temporal filter learned from previous observations of the same stimulus. Results of CCA applied to (B) a two-dimensional OU process projected onto one dimension, (C) a non-OU Gaussian process with a rational quadratic kernel, and (D) a contrast profile obtained by scanning a natural image at constant velocity. The first column shows example time series, the second column displays the canonical correlations ($\sigma_i$), and the third column shows the first two normalized temporal filters (canonical directions). In all numerical simulations, a small Gaussian white observation noise was added to the input signal.
• Figure 3: Dependence of past–future CCA results on observation noise in Gaussian processes. (A) Example time series generated from a Gaussian process with a rational quadratic kernel for high (blue) and low (red) SNR, varied through observation noise. (B) Correlation and mutual information between past projections onto canonical directions and the corresponding future signals. (C) Second canonical directions for high (blue) and low (red) SNRs. As observation noise increases, the filter shape transitions from multi-lobed to single-lobed. Shaded regions denote standard deviation across realizations. (D) Experimentally measured temporal filter of a retinal ganglion cell adapts to low contrast, which corresponds to lower SNR liu2015spike. These filters must pass through the origin (0,0), unlike the model filters (panel C), because physiological filtering is constrained to be both causal and continuous—conditions not enforced in our model.
• Figure 4: Experimentally measured and theoretically predicted responses to the staircase stimulus. (Top) Luminance as a function of time. (Left column) Experimental measurements of average neuronal activity via calcium imaging in three post-photoreceptor neurons in Drosophila: L1, L2 and L3 ketkar2022first. (Right column) Output from the first and second temporal filters derived from past-future CCA of natural images. Orange and pink traces differ because of variation in the SNR, i.e., observation noise level: $\sigma = 0.5, 0.05$ respectively.
• Figure 5: Experimentally observed and learned motion-detection networks exhibit similar response properties and synaptic connectivity. (A) The Drosophila ON motion-detection pathway takemura2017comprehensiveborst2023flies. For comparison with the model (Fig. \ref{['fig:fig0']}), we simplify the circuit by estimating the effective weights of inputs from L1 and L3 to T4 directly, as the intermediate neurons are thought to perform primarily contrast normalization matulis2020heterogeneous. (B) The unrectified response of a T4 analog in the trained ReSU network to a moving grating mirrors experimental results: the strongest, sharply peaked response occurs for an advancing bright (“ON”) edge in the preferred direction, whereas weaker responses to “OFF” edges in the null direction can be suppressed by thresholding. (C) Synaptically weighted contributions of each first-layer output channel to the T4 response (black) for ON-edge motion in the preferred and null directions around stimulus onset (dashed box in B). (D) Second-layer synaptic weights in the trained model reproduce the majority of sign patterns and approximate the relative amplitudes of synaptic inputs onto T4a in Drosophilatakemura2017comprehensive. Because experimental weights are based on synapse counts without knowledge of neuronal gain factors, we compare L1 and L3 inputs separately.