Dynamical similarity analysis can identify compositional dynamics developing in RNNs

TL;DR

This work tackles the challenge of benchmarking dynamic representational metrics by introducing two principled test cases based on attractor dynamics and compositional learning in RNNs. It compares three metrics—CKA, Procrustes, and Dynamic Similarity Analysis (DSA)—and demonstrates that DSA is more noise-robust and capable of linking evolving representations to computations, including in state-space models like Mamba where dynamics resemble reservoir behavior. The findings argue for a benchmark-driven approach to metric development, showing that DSA uniquely captures compositional dynamical motifs and their computational relevance, with implications for mechanistic interpretability and neuroscience-inspired analysis. Overall, the work provides a framework and initial evidence that dynamic metrics can steadily improve our understanding of how recurrent systems develop and deploy computations.

Abstract

Methods for analyzing representations in neural systems have become a popular tool in both neuroscience and mechanistic interpretability. Having measures to compare how similar activations of neurons are across conditions, architectures, and species, gives us a scalable way of learning how information is transformed within different neural networks. In contrast to this trend, recent investigations have revealed how some metrics can respond to spurious signals and hence give misleading results. To identify the most reliable metric and understand how measures could be improved, it is going to be important to identify specific test cases which can serve as benchmarks. Here we propose that the phenomena of compositional learning in recurrent neural networks (RNNs) allows us to build a test case for dynamical representation alignment metrics. By implementing this case, we show it enables us to test whether metrics can identify representations which gradually develop throughout learning and probe whether representations identified by metrics are relevant to computations executed by networks. By building both an attractor- and RNN-based test case, we show that the new Dynamical Similarity Analysis (DSA) is more noise robust and identifies behaviorally relevant representations more reliably than prior metrics (Procrustes, CKA). We also show how test cases can be used beyond evaluating metrics to study new architectures. Specifically, results from applying DSA to modern (Mamba) state space models, suggest that, in contrast to RNNs, these models may not exhibit changes to their recurrent dynamics due to their expressiveness. Overall, by developing test cases, we show DSA's exceptional ability to detect compositional dynamical motifs, thereby enhancing our understanding of how computations unfold in RNNs.

Figures (11)

• Figure 1: Schematic showing the idea of dynamic representations. A neural network (middle) is trained to solve a task which has a time course (left). We can now try to understand the internal mechanisms of that network by analyzing the representations of that network during task solving. For this, we record the activations of the network over time. These can be visualized by plotting the activation of all neurons as traces over time (right). These traces can now be compared across conditions. CKA and Procrustes only consider the geometric shape of the trace whereas new specialized metrics like DSA look at the traces as actual dynamics with momentum.
• Figure 2: DSA shows better noise robustness and identification of compositionally-combined dynamics. (a) Outline of model specifications to test both noise robustness (Model 1 vs. Model 2) and identification of combined dynamics (Model 2 vs. Model 3). (b) Results of noise robustness (c) compositional dynamics comparisons. All shadings are standard errors.
• Figure 3: DSA, but not CKA or Procrustes, captures how representations develop during compositional learning. (a) RNN inputs and outputs in our setup. (b) Schematic of dynamical representations during ‘Anti task’. Different colors show different task conditions. (c) Same as (b) but for the ‘Delay task’. (d) Same as (c) but showing the compound (Master) ‘DelayAnti task’. (e) Schematic of observation in driscoll2024flexible. (f) List of training conditions used in our test case. (g) We calculate the dissimilarity of all training groups to the ‘Master’ group with results shown by training schedule and metric. Color of boxplots refers to color used for training conditions in (f). Order at the bottom of the plot highlights the expected dissimilarity. Lines next to boxplots are standard errors.
• Figure 4: Only DSA can link developing representations to developing computations. (a) Schematic of analysis. As before, we calculate the dissimilarity between all training groups and the ‘Master’ group, but we now also capture the difference in accuracy between the two networks used for the dissimilarity calculation. We measure the dissimilarity and accuracy at multiple windows during training. (b) Results from analysis in (a) plotted by dissimilarity measure. All shadings are standard errors.
• Figure 5: DSA responds to gradual increase in task overlap during training. (a) Schematic showing how every training group is compared to every other training group to capture how the % of shared tasks during training affects dissimilarity. (b) Schematic showing how the analysis from prior section (Fig \ref{['fig:figure4']}a) is adapted to capture the share of pretraining tasks that a network has ‘experienced’ effects dissimilarity (called ‘rank’ to differentiate from terminology in Fig \ref{['fig:figure5']}a). Schematic only shows 10-25% group for visual simplicity, but all six time groups from Fig \ref{['fig:figure4']}a are analyzed. (c) DSA results of analysis in Fig \ref{['fig:figure5']}a. (d) DSA results of analysis in Fig \ref{['fig:figure5']}b. All shadings are standard errors.