The Topology and Geometry of Neural Representations

TL;DR

Analyzing functional MRI data and simulations, tRSA matches standard representational similarity analysis at identifying corresponding brain regions and model layers, while focusing on the core topological features distinguishing representations.

Abstract

A central question for neuroscience is how to characterize brain representations of perceptual and cognitive content. An ideal characterization should distinguish different functional regions with robustness to noise and idiosyncrasies of individual brains that do not correspond to computational differences. Previous studies have characterized brain representations by their representational geometry, which is defined by the representational dissimilarity matrix (RDM), a summary statistic that abstracts from the roles of individual neurons (or responses channels) and characterizes the discriminability of stimuli. Here we explore a further step of abstraction: from the geometry to the topology of brain representations. We propose topological representational similarity analysis (tRSA), an extension of representational similarity analysis (RSA) that uses a family of geo-topological summary statistics that generalizes the RDM to characterize the topology while de-emphasizing the geometry. We evaluate this new family of statistics in terms of the sensitivity and specificity for model selection using both simulations and fMRI data. In the simulations, the ground truth is a data-generating layer representation in a neural network model and the models are the same and other layers in different model instances (trained from different random seeds). In fMRI, the ground truth is a visual area and the models are the same and other areas measured in different subjects. Results show that topology-sensitive characterizations of population codes are robust to noise and interindividual variability and maintain excellent sensitivity to the unique representational signatures of different neural network layers and brain regions. These methods enable researchers to calibrate comparisons among representations in brains and models to be sensitive to the geometry, the topology, or a combination of both.

Figures (7)

• Figure 1: Comparing representations between brains and models. To understand the degree to which a computational model can account for the cognitive process of a certain brain region, the same set of stimuli is presented to both the model and the biological system. The response patterns across measured response channels (e.g. neurons or voxels) are then characterized by a summary statistic, the representational dissimilarity matrix (RDM, center), which defines the metric configuration of the stimuli in the neural population response space. However, the metric configuration can be sensitive to measurement noise and idiosyncrasies of individual brains that do not reflect computational function. An alternative summary statistic that captures the topology would be the adjacency matrix (right), which defines the unweighted graph of neighborhood relationships in the population response space. This summary statistic promises to be more robust to noise and idiosyncrasies, but may discard too much information. Considering the geometry (RDM) and topology (adjacency matrix) as extremes of a continuum suggests that it may be possible to get the best of both (Fig. 2).
• Figure 2: Intuition of the geo-topological transform of distances.(a) Consider the five visual stimuli from Fig. \ref{['fig1']}, whose representation in a visual cortical area can be characterized by its representational geometry. If the response patterns are all affected by isotropic noise (or the noise has been whitened by a transform), then the Euclidean distances monotonically reflect the discriminabilities. Variation among large distances, however, is not associated with great differences in discriminability, because all pairs of well-separated stimuli are nearly perfectly discriminable (dashed lines). Similarly, variation among very small distances is not associated with great differences in discriminability, because all pairs of neighboring stimuli are indiscriminable (thick black edge). This suggests that variation among small distances and variation among large distances can be suppressed in favor of emphasizing the transition from small to large distances. (b) To emphasize the transition from small to large distances while suppressing variation among small distances and variation among large distances, we can threshold the distances, such that small distances are pushed to zero and large distances are pushed to the maximum. We can either use a hard threshold (upper row) or a soft threshold (lower row). A hard threshold yields a binary matrix whose complement is the adjacency matrix of a graph that connects neighboring stimuli. A soft threshold creates a continuous transition, reflecting the graded increase in discriminability as the distance grows, and defines a weighted graph, where the weights reflect distances, but the pairs of stimuli that are furthest from each other are not directly connected (dashed lines in (a)).
• Figure 3: A family of geo-topological transforms of the RDM.(a) The geo-topological (GT) transform is formulated as a linear piecewise function, such that any distances smaller than a lower bound $l$ will be mapped to zero, and any distances bigger than an upper bound $u$ will be mapped to 1. Between $l$ and $u$, the transition is linear. (b) By varying the thresholds $l$ and $u$, we select among a family of GT transforms. (c) By applying different GT transforms to the RDM, we obtain so-called representational geo-topological matrices (RGTMs). (d) To interpret the way the GT transforms reflect geometric and topological properties of the representation, we group the family members in different zones of the plane spanned by $l$ and $u$. The closer a GT transform is to the upper left corner ($l=0, u=max$), the more similar the RGTM is to the RDM. As we approach the diagonal line ($l=u$), the GT transform approaches a hard threshold, emphasizing the topology rather than the geometry. As we move diagonally from the bottom left to the upper right, the RGTMs go from emphasizing the local neighbor relationships among the stimuli to emphasizing the global structure of the representation.
• Figure 4: The geometric and topological similarities between hypothetical neural representations (proof of concept).(a) We consider four hypothetical representations of 40 stimuli (balls in lower row of a). The response patterns are sampled from idealized continuous sets of neural response patterns (top row in a). Two of these continuous sets are manifolds (the untangled shapes) and the other two are not (the tangled shapes, where the set self-intersects, forming a neighborhood that is not homeomorphic to a Euclidean space). From left to right, we label the four representations the flat 8, the bent 8, the untangled flat 8, and the untangled bent 8. The flat 8 and the untangled flat 8 are geometrically similar, while being topologically dissimilar (with the former self-intersecting). Their bent versions, as well, are geometrically similar and topologically dissimilar. The flat 8 and the bent 8, on the other hand, are topologically similar (with the self-intersection creating two holes) and geometrically dissimilar (because the twisting substantially changes the metric geometry). Their untangled versions, as well, are topologically similar and geometrically dissimilar. (b) The RDMs (Euclidean distance, top row) do a good job characterizing the geometric relationships but are insensitive to the topological relationships. The RGTMs (middle row) are more sensitive to topology. To achieve local topological sensitivity, we chose $l=0$ and $u=0.075$, revealing the self-intersection in the two leftmost representations. The RGDMs (bottom row) are exquisitely sensitive to the topological relationships, while de-emphasizing geometrical relationships between the representations. Note the "eyes" in the two leftmost RGTMs, representing the self-intersections. Each RGDM captures the lengths of the shortest paths in the graph of the RGTM shown above it. The RGDMs more prominently reflect the topology as the "shortcut" paths enabled by the self-intersection affects shortest-path lengths for a broad swath of stimuli. (c) Multi-dimensional scaling (MDS) on the four representations shows the pairwise similarities among the four matrices of each row, confirming the increasing sensitivity to topological differences as we go from RDM to RGTM and on to RGDM.
• Figure 5: Brain-region identification accuracy across the family of geo-topological descriptors. Our analysis of human brain regions used fMRI data from 24 subjects, collected from 8 regions of interest, including the primary and secondary visual cortices, the lateral occipital complex, the occipital face area the fusiform face area, the parahippocampal place area, and the anterior temporal lobe. (a) Region-identification accuracy (RIA) is evaluated using leave-one-subject-out crossvalidation, where a classifier (on which we compute RIA) is trained on all available data except for one subject and then tested on that left-out subject. This process is repeated for each subject, with the final performance measure being the average across all iterations. We used bootstrapping to obtain an unbiased estimate of the standard error as the error bound for region identification accuracy and and used crossvalidation to prevent overfitting. We randomly sampled 10 sets of lower bounds $l$'s and upper bounds $u$'s in each of the five interpretable RGTM zones as defined in Fig. \ref{['fig3']} and applied a paired t-test to compare the RIA between different RGTM zones as defined in panel b) and Fig. \ref{['fig3']} (**** p < 1e-4, *** p < 1e-3, ** p < 1e-2, * p < 0.05). (b) RIA percentile (gray colorscale) as a function of the combination of upper and lower bounds, $l$ and $u$, defining the RGTM (layout as in Fig. \ref{['fig3']}) with the best-performing RGTM marked in red.