Variational Sampling of Temporal Trajectories

TL;DR

This work introduces a mechanism to learn the distribution of trajectories by parameterizing the transition function $f$ explicitly as an element in a function space.

Abstract

A deterministic temporal process can be determined by its trajectory, an element in the product space of (a) initial condition $z_0 \in \mathcal{Z}$ and (b) transition function $f: (\mathcal{Z}, \mathcal{T}) \to \mathcal{Z}$ often influenced by the control of the underlying dynamical system. Existing methods often model the transition function as a differential equation or as a recurrent neural network. Despite their effectiveness in predicting future measurements, few results have successfully established a method for sampling and statistical inference of trajectories using neural networks, partially due to constraints in the parameterization. In this work, we introduce a mechanism to learn the distribution of trajectories by parameterizing the transition function $f$ explicitly as an element in a function space. Our framework allows efficient synthesis of novel trajectories, while also directly providing a convenient tool for inference, i.e., uncertainty estimation, likelihood evaluations and out of distribution detection for abnormal trajectories. These capabilities can have implications for various downstream tasks, e.g., simulation and evaluation for reinforcement learning.

Figures (22)

• Figure 1: Structure of the model. First, Data Encoder is applied to temporal data sample to generate the initial point of the trajectory in latent space $\boldsymbol{z}_0$; At the same time, the Function Encoder (embedding module) is applied to generate parameterization $\theta$ of the sample from function space, $f_\theta$. The sample from function or embedding space, $f_\theta$, is used to describe together with a differential equation solver, to solve function DE and generate $\boldsymbol{z}_t$. Last, decoder is used to map latent space ODE stages into observed values.
• Figure 2: Function/embedding space, where each element is a DNN.
• Figure 3: The direct sampling of $\gamma$ from $N(0, 1)$ results in a bad reconstruction of trajectories. However, given the samples from the learned approximate posterior distribution $q(\gamma|\boldsymbol{x}^i)$, we can fit a better distribution on top of these sample, refereed as $S_\gamma$, to generate proper trajectories. The fitting is happened after we train our model.
• Figure 4: We present visualization of generated trajectories of three different data sets from MuJoCo. From left to right: Walker, Hopper, and 3 poles cartpole. Top row: Trajectories are generated by fixing initial condition $\boldsymbol{z}_0$ and sampling $\gamma \sim S_\gamma$, Bottom row: We transfer trajectory by sampling $\gamma$ from the exemplar, and applying it to $\boldsymbol{z}_0$. First row in each block represent original data from which we derive $\boldsymbol{z}_0$, second row is an exemplar used to transfer trajectory, and last is sampled/transferred.
• Figure 5: Row $1$ is a process used to derive $\boldsymbol{z}_0$, row $2$ is an exemplar, projection of which we would like to transfer, and third row is a result of transfer. Notice how the speed of row $2$ transfers to the first row, given an initial condition.