Adaptive control of recurrent neural networks using conceptors

TL;DR

Problem: Recurrent networks are typically adaptive during training but fixed during inference, limiting their ability to cope with changing conditions. Approach: the authors introduce an Adaptive Conceptor Control Loop (CCL) that online-estimates the current conceptor via autoconceptors and applies an adaptive projection $C_{adapt}$ to steer $x(k)$ toward a target subspace, i.e., $x(k+1) = C_{adapt}(k)[(1-\alpha)x(k) + \alpha\tanh(Wx(k) + W_{in}u(k) + b)]$. Key findings: the CCL yields more stable interpolation between temporal patterns than static conceptors, enhances robustness to partial degradation by preserving limit cycles, and supports distortion mitigation through a hierarchical Random Feature Conceptor (RFC) architecture. Significance: this framework extends RNN functionality beyond training, enabling on-the-fly adaptation in dynamic environments and offering potential for neuromorphic hardware implementations and few-shot learning scenarios.

Abstract

Recurrent Neural Networks excel at predicting and generating complex high-dimensional temporal patterns. Due to their inherent nonlinear dynamics and memory, they can learn unbounded temporal dependencies from data. In a Machine Learning setting, the network's parameters are adapted during a training phase to match the requirements of a given task/problem increasing its computational capabilities. After the training, the network parameters are kept fixed to exploit the learned computations. The static parameters thereby render the network unadaptive to changing conditions, such as external or internal perturbation. In this manuscript, we demonstrate how keeping parts of the network adaptive even after the training enhances its functionality and robustness. Here, we utilize the conceptor framework and conceptualize an adaptive control loop analyzing the network's behavior continuously and adjusting its time-varying internal representation to follow a desired target. We demonstrate how the added adaptivity of the network supports the computational functionality in three distinct tasks: interpolation of temporal patterns, stabilization against partial network degradation, and robustness against input distortion. Our results highlight the potential of adaptive networks in machine learning beyond training, enabling them to not only learn complex patterns but also dynamically adjust to changing environments, ultimately broadening their applicability.

Figures (9)

• Figure 1: Recurrent neural network subject to three different input time series with different frequencies and amplitudes (color-coded). On the right side, we show the affine projection of the network dynamics related to the three different input patterns onto a low-dimensional manifold using principal component analysis (here using the two largest principal components).
• Figure 2: Conceptor control loop where the current conceptor of the network $C$(k) is measured via the autoconceptor framework and slightly linearly adapted towards the target conceptor $C_{target}$. The adapted conceptor $C_{adapt}(n)$ is then applied to the network. Thereby, the dynamics of the network are pushed towards a linear subspace that are close to the targeted subspace.
• Figure 3: a) Autonomously generated output $y(k)$ of the conceptor-RNN trained on two sinusoidal time series with $T_0=20$ and different $T_1\in[25,30,35]$ (different blue shades). Along the x-axis the plugged in conceptor is a linear interpolation between the conceptors $C_0$ and $C_1$ defined during training. b) Period of the intermediary solutions while scanning the interpolation parameter $\lambda$ in the range 0 to 1. c) Output of the conceptor-RNN with applied conceptor control loop during interpolation between the two conceptors $C_0$ and $C_1$ generated for different periods of the training time series given by two sinusoidal time series with $T_0=20$ and different $T_1\in[25,30,35]$ (different red shades). Similar to a) and b), along the x-axis the interpolation parameter $\lambda$ is scanned in the range 0 to 1. d) Varying period of the autonomously generated time series $y(k)$ during the interpolation while applying the adaptive conceptor control loop.
• Figure 4: Period of the output patterns generated by a RNN with applied conceptor control loop to interpolate sine wave time series with different input periods $T_0=20$ and $T_1\in[37.5,50]$.
• Figure 5: a) Projection of the output of the RNN trained to predict a 94-dimensional time series motion capture time series of a human running behavior. The RNN is degraded by removing a random subset of 200 of its 1500 neurons. The trajectory is depicted for the RNN with the conceptor control loop (blue), compared with only the static conceptor (red), and with a baseline where the RNN is not degraded (green) along the three first principal components of the output. b) The same output generation (same color code) is represented with the geometry of the human body angles. The number of steps depicted is the same for all the conditions and is chosen to showcase one full cycle of the baseline (green).