Reconstructing shared dynamics with a deep neural network

TL;DR

The paper addresses extracting a hidden shared driver from time-series data generated by two observed chaotic subsystems. It introduces a Mapper-Coach neural network that jointly learns a mapping from a reconstructed state to the latent driver $Z$ (Mapper) and a predictor for the observed dynamics (Coach), trained end-to-end so that the bottleneck activity encodes the shared input. On a coupled logistic-map example, the approach achieves high predictive accuracy for $x_t$ ($r^2\approx0.99$) and strong latent reconstruction ($r^2\approx0.97$) with the latent signal correlating to the bottleneck output. This demonstrates a data-driven route to reveal fast, continuous hidden drivers in dynamical systems and suggests broad applicability to settings where direct intervention or access to the full state is impossible.

Abstract

Determining hidden shared patterns behind dynamic phenomena can be a game-changer in multiple areas of research. Here we present the principles and show a method to identify hidden shared dynamics from time series by a two-module, feedforward neural network architecture: the Mapper-Coach network. We reconstruct unobserved, continuous latent variable input, the time series generated by a chaotic logistic map, from the observed values of two simultaneously forced chaotic logistic maps. The network has been trained to predict one of the observed time series based on its own past and conditioned on the other observed time series by error-back propagation. It was shown, that after this prediction have been learned successfully, the activity of the bottleneck neuron, connecting the mapper and the coach module, correlated strongly with the latent shared input variable. The method has the potential to reveal hidden components of dynamical systems, where experimental intervention is not possible.

Figures (8)

• Figure 1: Shared dynamics reconstruction from time series. The shared latent driver ($Z$) leaves similar mark (green) on the forced subsystems ($X$ and $Y$). This redundancy can be exploited to extract the latent part of the dynamics from time series observations of the forced parts using artificial neural networks.
• Figure 2: Directed graph representation of the causal connections in the coupled logistic map system. The $z$ variable (red) is a hidden common cause of the two observed variables, $x$ and $y$ (blue).
• Figure 3: Dynamics on the reconstructed state space. The next value $y_{t+1}$ and state can be written as a function of current reconstructed state following the black arrows from $Y_t$ to $Y_{t+1}$. The composition of mappings along the route realize the $\Tilde{F}$ dynamics on the reconstructed space. The $\Phi$ embedding function is invertible, however the dynamics $F$ is not invertible, neither the observation function $g$.
• Figure 4: The return map of the $y$ variable. The distinct fibers corresponds to different values of the latent variable $z$. Due to the coupling and the reflecting boundary conditions, there is a small self-intersecting area in this 2D embedding, where the value of the latent is ambiguous.
• Figure 5: The architecture of the mapper-coach network. The feedforward neural network predicts present values of $x$. One cannot predict accurately $x(t)$ values based on solely $x(t-1)$, because the lack of information about $z(t-1)$. This missing information is sipped out from $Y(t)$ through the red bottleneck At the training phase the full mapper-coach network can be trained with backpropagation algorithm and -- after successful training -- the activity of the red hidden unit has a one-to-one correspondence with values of the latent $z$ variable. At this reconstruction stage, the coach can be detached from the mapper, which latter estimates the mapping ($\phi$) from the reconstructed state space of the effect ($Y$) to the state space of the hidden variable ($z$).