An Overlooked Role of Context-Sensitive Dendrites

TL;DR

The paper reframes learning in neural systems by emphasizing context-sensitive processing in dendritic contexts, introducing cooperative CS-TPNs that integrate apical context with feedforward signals via a MOD function and thalamic gating (U). Coupled with a BDSP learning rule, CS-TPNs exhibit faster, local online learning with substantially fewer neuronal events, demonstrated on both a shallow XOR task and a 50-layer AV CNN. The results suggest universality across network types and learning rules, achieving major reductions in neurons and compute (MACs/FLOPs) while maintaining or improving performance on audiovisual speech tasks. This work implies a scalable, biologically grounded path to more efficient AI systems on neuromorphic substrates and real-world multimodal data processing.

Abstract

To date, most dendritic studies have predominantly focused on the apical zone of pyramidal two-point neurons (TPNs) receiving only feedback (FB) connections from higher perceptual layers and using them for learning. Recent cellular neurophysiology and computational neuroscience studies suggests that the apical input (context), coming from feedback and lateral connections, is multifaceted and far more diverse, with greater implications for ongoing learning and processing in the brain than previously realized. In addition to the FB, the apical tuft receives signals from neighboring cells of the same network as proximal (P) context, other parts of the brain as distal (D) context, and overall coherent information across the network as universal (U) context. The integrated context (C) amplifies and suppresses the transmission of coherent and conflicting feedforward (FF) signals, respectively. Specifically, we show that complex context-sensitive (CS)-TPNs flexibly integrate C moment-by-moment with the FF somatic current at the soma such that the somatic current is amplified when both feedforward (FF) and C are coherent; otherwise, it is attenuated. This generates the event only when the FF and C currents are coherent, which is then translated into a singlet or a burst based on the FB information. Spiking simulation results show that this flexible integration of somatic and contextual currents enables the propagation of more coherent signals (bursts), making learning faster with fewer neurons. Similar behavior is observed when this functioning is used in conventional artificial networks, where orders of magnitude fewer neurons are required to process vast amounts of heterogeneous real-world audio-visual (AV) data trained using backpropagation (BP). The computational findings presented here demonstrate the universality of CS-TPNs, suggesting a dendritic narrative that was previously overlooked.

Figures (5)

• Figure 1: Signal propagation and burst control in TPNs combined with short-term plasticity. Here, we show that the spiking neural network composed of CS-TPNs significantly reduces the required number of events, including both singlets and bursts, for the task at hand compared to simple TPNs. a Context-insensitive TPNs with BDSP. The contextual current ($I_c$) does not influence the somatic potential; therefore, the learning is inspired by TPNs, but the processing is not, and is driven by point neuron conception. (b) Context-insensitive TPNs signal propagation and burst control payeur2021burst. The circuit has two populations of neurons (Pop 1, bottom and Pop 2, top), 4000 neurons each. The neurons in Pop 1 receive external input as somatic current $I_s$ and those in Pop 2 receive dendritic current $I_d$. The feedforward pass (soma to soma) is from Pop 1 to Pop 2 through short-term depression (STD) synapses similar to payeur2021burst. The output from Pop 1 is also projected to a population providing disynaptic inhibition (disk). The feed-backward pass (soma to dendritic compartment) is from Pop 2 to Pop 1 through the short-term facilitaion (STF) synapse. Similarly, the output from Pop 2 soma is connected to a population providing disynaptic inhibition (square). c Context-insensitive TPNs with BDSP, receiving FB as well as P, D, and U. Both learning and processing are influenced by the contextual current. The contextual current controls the membrane potential in a way that events occur only when the contextual current and somatic current are coherent. This is due to dense coordination and cooperation among TPNs, amplifying the transmission of coherent information and suppressing the transmission of incoherent information. (d) Context-sensitive TPNs signal propagation and burst control. Neurons in both populations receive P, however, the D and U signals are set to zero for simplicity. The goal here is to test the context-sensitive operation and for that P is sufficient. Specifically, P is the contextual information coming from the neighbouring neurons within the same population. Please note that both populations have the same CS-TPNs dynamics. However, P, D, and U are included in the XOR simulations in Fig 2. It can be seen that the response of BP and ER at apical and soma, respectively, for both standard TPNs and CS-TPNs, after injecting $I_s$ in Pop 1, and $I_d$ in Pop 2 show different behaviour. Specifically, the CS-TPNs require fewer events, including singlets and bursts, to represent the input sinusoidal signal.
• Figure 2: (a) CMI-inspired cooperative CS-TPNs + BDSP for XOR problem. Individual TPNs receive FB, P, D, and U inputs to conditionally segregate the coherent and incoherent FF signals, respectively. In terms of their sequence, first, coherent and incoherent signals are segregated by the TPNs. Then, these coherent signals are recombined by PNs, extracting synergistic FF components from all the coherent multistreams. This happens with the help of an additional ensemble representing U with a population of 50 PNs. U is broadcasted to TPNs along with the current local context adeel2020consciousmuckli_2023_8380094adeel2023unlocking. (b) Impact of this information processing mechanism: an increased speed in 'local' and 'online' learning and processing can be observed when P and D are integrated in CS TPNs and when P, D, and U are integrated in CS TPNs, compared to BDSP alone (c) BDSP XOR output reconstruction in 250 epochs (d) BDSP + P + D XOR output in 250 epochs (e) BDSP + P + D + U XOR output reconstruction in 250 epochs. Note that cooperative CS TPNs + BDSP cross the target threshold faster with the same number of neurons.
• Figure 3: (a) Raster plots with 150 neurons: TPNs + BDSP (i) cooperative CS-TPNs + BDSP (ii). CS-TPNs tend to remain largely silent when information is less relevant but become active (bursting) when information is relevant, compared to BDSP alone. Note the clear distinction between coherent (burst) and conflicting signals (singlet). It implies that CS-TPNs are bursting far more (transmitting coherent information) than singlets (transmitting incoherent information), hence learn faster. The bottom blue line reflect event rates. (b) it is notable that TPNs fire more frequently than CS-TPNs. (c) A 50-layer deep CNN composed of CS-TPNs requires significantly fewer neurons at any time during training with better generalization capability (Table II) compared to a deep CNN composed of PNs. This reveals the universal applicability of CS-TPN-driven efficient information processing regardless of the learning mechanism type. See: http://cmilab.org/research/
• Figure 4: A 50-layer deep CNN composed of CS-TPNs requires significantly fewer neurons at any time during training with better generalization capability (Table II) compared to a deep CNN composed of PNs. This reveals the universal applicability of CS-TPN-driven efficient information processing regardless of the learning mechanism type.
• Figure 5: Effects of different time scales on the XOR task for the population size of 200 neurons on Standard BDSP and CS-TPNs+BDSP.a and c, Comparison of costs for different duration of examples T (in s). T = 8s (green line) is the duration used in Fig. 2. b and d, Output event rate (ER) after learning for the cases in a,c classifying between 'true(1)' and 'false(o)' for the XOR. The costs for the standard network (a) reach a minimum of $1.5$ starting at epoch 250. The context-sensitive network attains the cost of $1.0$ starting at epoch $120$. It is evident from the graphs that the FF context-sensitive network learns faster and better.