Mapping Connectomic Structure to Function(s) in Cerebellar-like Networks using Kernel Regression

TL;DR

This work addresses how nonrandom connectivity in cerebellar-like networks shapes learning by reframing the circuitry as kernel regression with a fixed nonlinear expansion. By modeling expansion-layer weights with a covariance ${m{Sigma}}$ that encodes biased or grouped connectivity, the authors derive how connectivity alters the kernel and its eigenstructure, and thus the inductive bias via learnability of eigenfunctions $v_i({m{x}})$ with eigenvalues $\lambda_i$ (where ${\mathcal L}(v_i)=\lambda_i/(\lambda_i+\kappa)$). They demonstrate that overconnection to specific inputs biases learning toward functions that vary along that input, while grouping biases the network toward learning functions that generalize across a connected group, with analogous effects shown in both analytic toy models and more realistic sparse setups. The results offer a mathematically tractable bridge between connectomic structure and functional learning, yielding testable predictions about how wiring motifs influence classification tasks in cerebellar-like circuits. Overall, the paper provides a normative account linking anatomy to computation, guiding future experiments and theoretical work on processing motifs in cerebellar-like networks.

Abstract

Cerebellar-like networks, in which input activity patterns are separated by projection to a much higher-dimensional space before classification, are a recurring neurobiological motif, present in the cerebellum, dentate gyrus, insect olfactory system, and electrosensory system of the electric fish. Their relatively well-understood design presents a promising test-case for probing principles of biological learning. The circuits' expansive projections have long been modelled as random, enabling effective general purpose pattern separation. However, electron-microscopy studies have discovered interesting hints of structure in both the fly mushroom body and mouse cerebellum. Recent numerical work suggested that this non-random connectivity enables the circuit to prioritise learning of some, presumably natural, tasks over others. Here, rather than numerical results, we present a robust mathematical link between the observed connectivity patterns and the cerebellar circuit's learning ability. In particular, we extend a simplified kernel regression model of the system and use recent machine learning theory results to relate connectivity to learning. We find that the reported structure in the projection weights shapes the network's inductive bias in intuitive ways: functions are easier to learn if they depend on inputs that are oversampled, or on collections of neurons that tend to connect to the same hidden layer neurons. Our approach is analytically tractable and pleasingly simple, and we hope it continues to serve as a model for understanding the functional implications of other processing motifs in cerebellar-like networks.

Figures (7)

• Figure 1: A) Schematic of fly mushroom body circuit. Odorants trigger activity in olfactory receptor neurons (ORNs). ORNs contain a unique receptor protein, signalled by their colour; neurons with the same receptor protein send projections to a shared glormulus. There they synapse onto projection neurons, which carry the activity to the mushroom body. In the mushroom body the dimensionality of the representation is expanded by a factor of 50 in the kenyon cells (KCs), before being assigned a valence by mushroom body output neurons (MBONs). Dopaminergic neurons encode an error signal that modulates the KC-to-MBON connectivity to ensure correct classification. B) Simplified Network We use a model focusing only on the expansive projection and subsequent labelling which (C) has uncanny similarities with the kernel regression algorithm. We consider two structural motifs - (D) biased connectivity, in which some input neurons connect more than others; and (E) grouped connectivity, in which the inputs are grouped: if a hidden layer neuron is connected to one member of the group it is likely to be connected to others.
• Figure 2: A) A single linear layer can only classify linearly seperable data. B) However adding a fixed first layer of nonlinear processing can permit a linear readout layer to perform nonlinear classifications. C) Given any finite dataset there are infinitely many possible generalisations to unseen data. Without prior assumptions there is no reason to choose between them, i.e. from training data alone both curves are equally reasonable. D) Generic prior assumptions might prefer a smooth solution, while prior knowledge on how temperature varies throughout the year would select a solution with a period of 12 months.
• Figure 3: Consider a kernel defined on a circle, points that are nearby are similar, as shown by the red curve which measured similarity between each angle $\theta$ and $\theta_0$. It is largest at $\theta_0$ and decays further away. An eigenfunction that constructively interferes with the kernel similarity, green, with have a large eigenvalue, while destructive interference, purple, produces a small eigenvalue.
• Figure 4: A) We sample each expansion layer weight vector, ${\bm{J}}_i$, independently. B) We model two structural motifs: 1) biased connectivity, in which some input neurons connect more than others; and 2) and grouped connectivity, in which the inputs are grouped; i.e., if a hidden layer neuron is connected to one member of the group it is likely to be connected to others. C) We build an analytically tractable model of these structures using multivariate Gaussians. Random connectivity is modelled by sampling each weight from a Gaussian with mean zero and variance 1; biased connectivity is implemented by increasing the variance of one dimension; grouping is implemented by correlating each hidden neuron's connection to members of the input neuron grouping, meaning that if it has a strong positive connection to one member of the group, it likely will to the others.
• Figure 5: Each eigenfunction maps from points on the sphere to a label, which we display as a colour, left. To display the full function, we project to 2D space (for map nerds, we use the equirectangular projection), and show the label as a function of latitude and longitude. For ease of orientation, the four of the points lie on the $x_1$ and $x_2$ neuron axes are shown by the red and blue points.