Synaptic Activation and Dual Liquid Dynamics for Interpretable Bio-Inspired Models

TL;DR

Interpreting gated temporal neural networks in safety-critical settings remains challenging. The paper unifies bio-inspired RNNs by deriving Liquid-Capacitance ($LC$) and Liquid-Resistance + Capacitance ($LRC$) models from Electrical-Equivalent Circuits (EECs), exploring neural activation ($NA$) and synaptic activation ($SA$). It evaluates these models on a lane-keeping task with imitation learning, measuring validation loss, neural activity correlations with road trajectories, saliency maps, and attention robustness via SSIM, finding that $LRC$ variants—especially $LRC$-$SA$—achieve lower validation loss and more interpretable dynamics. This work provides a principled, interpretable bio-inspired framework for dense RNN policies and lays groundwork for concept learning in safety-critical temporal tasks.

Abstract

In this paper, we present a unified framework for various bio-inspired models to better understand their structural and functional differences. We show that liquid-capacitance-extended models lead to interpretable behavior even in dense, all-to-all recurrent neural network (RNN) policies. We further demonstrate that incorporating chemical synapses improves interpretability and that combining chemical synapses with synaptic activation yields the most accurate and interpretable RNN models. To assess the accuracy and interpretability of these RNN policies, we consider the challenging lane-keeping control task and evaluate performance across multiple metrics, including turn-weighted validation loss, neural activity during driving, absolute correlation between neural activity and road trajectory, saliency maps of the networks' attention, and the robustness of their saliency maps measured by the structural similarity index.

Figures (8)

• Figure 1: Structure of CT-RNNs and LTCs (top) and LC and LRC (bottom) cells, respectively. In addition to this, we further distinguish LC-NA and LC-SA, and LRC-NA and LRC-SA, respectively, to measure the effect of neural vs synaptic activation, which is incorporated in their $u$ and $f$ terms.
• Figure 2: A neural network with either neural activation (NA) on the top, or synaptic activation (SA), on the bottom. While the NA model associates the activation $\varphi_j$ to the pre-synaptic neuron $h_j$ itself (by assuming that all outgoing synapses of neuron $h_j$ have the same dynamics), the SA model assumes different dynamics, by using separate activations $\{\varphi_{j,i}, ..., \varphi_{j,i+l}\}$ for each outgoing synapse from pre-synaptic $h_j$ to post-synaptic neurons $h_i$ to $h_{i+l}$.
• Figure 3: Left: Simulation of autonomous driving in diverse seasons. The network's input is highlighted by the red rectangle, and its steering prediction is depicted by the blue spline. The car's position on the road is shown on the right side of the frames. Right: Starting from the front camera image, features are extracted using a CNN head and passed to an RNN policy to predict the steering angle. Training and validation are performed in an open-loop setting without the feedback connection (dashed line), which is only enabled during simulator testing.
• Figure 4: Robustness of the attention, measured by the Structural Similarity Index (SSIM) of the models, in summer (left) and in winter (right). The LRC-based models maintain the most similar focus of their attention in the presence of noise, indicated by the boxplots closer to 1. Lighter color refers to additional Gaussian noise of zero-mean and $\sigma^2=0.1$ variance, and darker to $\sigma^2=0.2$ variance.
• Figure 5: Sigmoidal-activation functions learned for Neuron 1 of the Lane-Keeping policy of LRC-NA and LRC-SA. As one can see, Neuron 1 in LRC-SA takes advantage of its additional flexibility to learn a different dynamics for each of its outgoing synapses.