Learning minimal representations of stochastic processes with variational autoencoders

TL;DR

An unsupervised machine learning approach to determine the minimal set of parameters required to effectively describe the dynamics of a stochastic process and enables the autonomous discovery of unknown parameters describing stochastic processes, hence enhancing the comprehension of complex phenomena across various fields.

Abstract

Stochastic processes have found numerous applications in science, as they are broadly used to model a variety of natural phenomena. Due to their intrinsic randomness and uncertainty, they are, however, difficult to characterize. Here, we introduce an unsupervised machine learning approach to determine the minimal set of parameters required to effectively describe the dynamics of a stochastic process. Our method builds upon an extended $β$-variational autoencoder architecture. By means of simulated datasets corresponding to paradigmatic diffusion models, we showcase its effectiveness in extracting the minimal relevant parameters that accurately describe these dynamics. Furthermore, the method enables the generation of new trajectories that faithfully replicate the expected stochastic behavior. Overall, our approach enables the autonomous discovery of unknown parameters describing stochastic processes, hence enhancing our comprehension of complex phenomena across various fields.

Figures (5)

• Figure 1: Interpretable autoregressive $\beta$-VAE. Given the displacements $\mathbf{\Delta x}(t)$ of a diffusion trajectory, the encoder (orange) compresses them into an interpretable latent space (blue), in which few neurons (dark blue) represent physical features of the input data while others are noised out (light blue). An autoregressive decoder (green) generates from this latent representation the displacements $\Delta \mathbf{x'}(t)$ of a new trajectory recursively, considering a certain receptive field RF (light green cone).
• Figure 2: Interpretation of the latent space. Distribution of latent neuron activations $z_i$ for four datasets: (a) BM, (b, c) FBM, (d, e) SBM, and (f) BM with confinement. Only surviving neurons are shown (i.e., $\sigma_{z_i} \ll 1$). For all datasets, the number of surviving neurons agrees with their respective number of degrees of freedom.
• Figure 3: Statistical properties of generated anomalous diffusion. Top (bottom) row corresponds to the FBM (SBM) dataset. (a) Displacements correlations $C=\left|\langle \Delta x_t \Delta x_{t+\Delta t} \rangle\right|/\Delta x_0^2$ for the input (dotted) and generated data (solid) with $\alpha=0.6, 1.8$ (blue and green, respectively). (b) Anomalous exponent $\alpha_g$ of the generated FBM data fitted from the time-averaged mean squared displacement at different $\Delta t$ for different input $\alpha$. Insets show the two-dimensional histograms of the input vs. generated anomalous exponent at the highlighted $\Delta t$, before (blue) and after (orange) the receptive field RF. (c) Anomalous exponent $\alpha_g$ of the generated SBM data vs. the input exponent $\alpha$ for various $D_0$. (d) Evolution of the diffusion coefficient for generated SBM trajectories at various $\alpha$. Dotted lines show the expected scalings.
• Figure 4: Relation between the loss and the diffusion parameters. Negative log-likelihood (NLL) of two models, trained with: (a) FBM and (b) SBM trajectories as a function of the trajectories' anomalous exponent $\alpha$. Colors encode the diffusion coefficient $D$ and $D_0$ for FBM and SBM respectively.
• Figure 5: 3D representation of the latent space. Distribution of latent neurons activations $z_i$ w.r.t. the anomalous exponent $\alpha$ and the diffusion coefficient, $D$ for FBM (left column) and $D_0$ for SBM (right column). $z_1$ and $z_2$ show a clear relation w.r.t. to the diffusive parameters, while $z_3$ has been noised out and is completely uninformative.