What Representational Similarity Measures Imply about Decodable Information

TL;DR

This work demonstrates a tight link between the geometry of neural representations and the ability to linearly decode information and suggests new ways of measuring similarity between neural systems and also provides novel, unifying interpretations of existing measures.

Abstract

Neural responses encode information that is useful for a variety of downstream tasks. A common approach to understand these systems is to build regression models or ``decoders'' that reconstruct features of the stimulus from neural responses. Popular neural network similarity measures like centered kernel alignment (CKA), canonical correlation analysis (CCA), and Procrustes shape distance, do not explicitly leverage this perspective and instead highlight geometric invariances to orthogonal or affine transformations when comparing representations. Here, we show that many of these measures can, in fact, be equivalently motivated from a decoding perspective. Specifically, measures like CKA and CCA quantify the average alignment between optimal linear readouts across a distribution of decoding tasks. We also show that the Procrustes shape distance upper bounds the distance between optimal linear readouts and that the converse holds for representations with low participation ratio. Overall, our work demonstrates a tight link between the geometry of neural representations and the ability to linearly decode information. This perspective suggests new ways of measuring similarity between neural systems and also provides novel, unifying interpretations of existing measures.

Figures (2)

• Figure 1: Schematic of the proposed framework for comparing representations $\boldsymbol{X}$ and $\boldsymbol{Y}$ (each dot represents mean neural responses to one of $M$ conditions) in terms of a decoding target $\boldsymbol{z}$. First, optimal linear decoding weights $\boldsymbol{w}^*$ and $\boldsymbol{v}^*$ are computed. Then the similarity between the two systems is measured in terms of the decoding similarity: $\langle \boldsymbol{X} \boldsymbol{w}^*, \boldsymbol{Y} \boldsymbol{v}^* \rangle$.
• Figure 2: Bounds in \ref{['eq:bound_proc']} plotted (solid lines) for varying participation ratio $\mathcal{R}_\Delta$ with $\alpha = \beta = 1$. A) Allowed regions (between the solid curves of like color) of Procrustes distance and expected Euclidean distance between decoded signals for different values of participation ratio $\mathcal{R}_\Delta$. B--D) Allowed regions for particular $\mathcal{R}_\Delta$ intervals (black solid lines) populated with calculated distances between pairs of randomly sampled positive semi-definite (PSD) matrices of size $50 \times 50$, subsampled to those that have $\mathcal{R}_\Delta$ in the particular interval (colored points). Different colors represent different random ensembles of positive semi-definite matrix pairs, which are described in \ref{['appendix:fig2']}.

Theorems & Definitions (10)

• Proposition 1
• Proposition 2
• proof : Proof of \ref{['proposition:best-worst-case']}
• proof : Proof of \ref{['proposition:avg-case']}
• Corollary 1
• Proposition 3
• Lemma 1
• proof
• Lemma 2
• proof