Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Binding in hippocampal-entorhinal circuits enables compositionality in cognitive maps

Christopher J. Kymn, Sonia Mazelet, Anthony Thomas, Denis Kleyko, E. Paxon Frady, Friedrich T. Sommer, Bruno A. Olshausen

Abstract

We propose a normative model for spatial representation in the hippocampal formation that combines optimality principles, such as maximizing coding range and spatial information per neuron, with an algebraic framework for computing in distributed representation. Spatial position is encoded in a residue number system, with individual residues represented by high-dimensional, complex-valued vectors. These are composed into a single vector representing position by a similarity-preserving, conjunctive vector-binding operation. Self-consistency between the representations of the overall position and of the individual residues is enforced by a modular attractor network whose modules correspond to the grid cell modules in entorhinal cortex. The vector binding operation can also associate different contexts to spatial representations, yielding a model for entorhinal cortex and hippocampus. We show that the model achieves normative desiderata including superlinear scaling of patterns with dimension, robust error correction, and hexagonal, carry-free encoding of spatial position. These properties in turn enable robust path integration and association with sensory inputs. More generally, the model formalizes how compositional computations could occur in the hippocampal formation and leads to testable experimental predictions.

Binding in hippocampal-entorhinal circuits enables compositionality in cognitive maps

Abstract

We propose a normative model for spatial representation in the hippocampal formation that combines optimality principles, such as maximizing coding range and spatial information per neuron, with an algebraic framework for computing in distributed representation. Spatial position is encoded in a residue number system, with individual residues represented by high-dimensional, complex-valued vectors. These are composed into a single vector representing position by a similarity-preserving, conjunctive vector-binding operation. Self-consistency between the representations of the overall position and of the individual residues is enforced by a modular attractor network whose modules correspond to the grid cell modules in entorhinal cortex. The vector binding operation can also associate different contexts to spatial representations, yielding a model for entorhinal cortex and hippocampus. We show that the model achieves normative desiderata including superlinear scaling of patterns with dimension, robust error correction, and hexagonal, carry-free encoding of spatial position. These properties in turn enable robust path integration and association with sensory inputs. More generally, the model formalizes how compositional computations could occur in the hippocampal formation and leads to testable experimental predictions.
Paper Structure (28 sections, 1 theorem, 20 equations, 12 figures)

This paper contains 28 sections, 1 theorem, 20 equations, 12 figures.

Table of Contents

  1. Introduction
  2. A normative model for the hippocampal formation
  3. Principles for representing space
  4. Modular attractor network for spatial representation
  5. Mapping the model to the HF
  6. Coding properties of the model
  7. RNS representations have exponential coding range
  8. The modular attractor network has superlinear coding range
  9. Robust error correction
  10. Interpolation between patterns enables continuous path integration
  11. Triangular frames in 2D maximize spatial information
  12. Testing functionalities of the model
  13. Robust path integration
  14. Denoising sensory states via a heteroassociative memory
  15. Discussion
  16. ...and 13 more sections

Key Result

Theorem 1

Fix any $D > 0$ and $\delta \in (0,1)$. For any pair $a,a' \in {\mathbb{R}}$ such that $a - a'$ is not an integer multiple of $m$, with probability at least $1-\delta$ over randomness in the draw of $k_{1},...,k_{D}$:

Figures (12)

  • Figure 1: Schematic of proposed attractor model. In MEC, the $\mathbf{g}_i$ are residue representations in grid modules, and c encodes a context label. Input of velocity estimate $\mathbf{q}(\mathbf{v})$ can produce path integration in grid modules via binding, denoted by $\odot$. In HC, p represents contextualized place. Binding serves two roles in the MEC/HC interaction (symbolized by bidirectional arrows): a) factorizing p into $\mathbf{g}_i$'s, and b) generating an update of p from the $\mathbf{g}_i$'s, for example, after path integration. In LEC, s represents sensory input, interacting with p through a learned heteroassociative projection.
  • Figure 2: Residue number systems, combined with a modular attractor network (resonator network), result in a new kind of attractor neural network with favorable scaling for a large combinatorial range.A) Number of encoding states, $M$, grows rapidly in the number of modules, up to a maximum established by Landau's function (black dots). B) Coefficient of coding range, M, scales roughly as $\mathcal{O}(D^{\alpha_K})$, depending on the number of moduli, $K$, but with $\alpha_K > 1$. C) Estimation of scaling from slopes of linear regression (fit to log-log scale). Higher values of $K$ require a higher dimension to achieve a particular coding range; empirical values are close to $\alpha_K = \frac{K}{K-1}$.
  • Figure 3: Recovery of encoded positions is robust to various sources of noise. A) Visualization of the von Mises weight distribution. Note that the magnitude of the noise is inversely proportional to $\kappa$, and that the variance of the phase perturbation is much larger than the distance between the discrete states of phasors. B-D) Visualizations of accuracy as a function of coding range and $\kappa$ for three separate cases: input noise (B), update noise (C), and codebook noise (D). Cases are shown in order of increasing difficulty. The resonator network maintains perfect accuracy up to a point, after which accuracy decays at an earlier point than the noiseless dynamics (black curve).
  • Figure 4: Continuity of attractor landscape enables sub-integer decoding and path integration.A) Visualization of interpolation between two integer states. The position of the fractional value can be estimated by fitting a periodic sinc function (Appendix \ref{['sec:residue_kernel']}) based on the inner products with integer codebooks (visualized in dots), then finding the location of the peak. B, C) Sub-integer states can be be decoded, up to a precision set by the noise level. Note that in both cases, sub-integer decoding can be just as accurate as integer decoding for the same range, even though the sub-integer decoding problem is strictly harder. Even $\kappa = 4$ is sufficient to achieve accuracy within a precision of $\Delta x = 0.07$, but for higher noise ($\kappa = 2$), the precision is worse. D) The best spatial precision (in bits) that can be decoded for a fixed noise level. Less noise achieves both a higher coding range and higher information content per vector.
  • Figure 5: Hexagonal coding improves spatial resolution.A) Voronoi tessellation for $m=5$. Each distinct color corresponds to a unique codeword in $\mathbb{C}^{D}$. Black arrows show the coordinate axes of the triangular 'Mercedes-Benz' frame in 2D. B) Hexagonal lattices have higher entropy than square lattices, allowing each state to carry higher resolution in its spatial output.
  • ...and 7 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof