Learning to Approximate Particle Smoothing Trajectories via Diffusion Generative Models

TL;DR

This work tackles learning dynamical systems from sparse observations by marrying conditional particle filtering with ancestral sampling (CPF-AS) and diffusion-based generative modeling through Schrödinger-bridge concepts. It first uses CPF-AS to generate smoothing trajectories that respect observed marginals and terminal constraints, then learns a neural SDE drift that reproduces these trajectories for efficient sampling. The approach yields a scalable, conditional generative model that can produce high-quality trajectories under complex constraints, demonstrated across synthetic, vehicle-tracking, and single-cell RNA datasets. By bridging particle smoothing with diffusion-model learning, the method enables rapid conditional trajectory generation and offers insights into system behavior under partial observability and strong marginal requirements.

Abstract

Learning dynamical systems from sparse observations is critical in numerous fields, including biology, finance, and physics. Even if tackling such problems is standard in general information fusion, it remains challenging for contemporary machine learning models, such as diffusion models. We introduce a method that integrates conditional particle filtering with ancestral sampling and diffusion models, enabling the generation of realistic trajectories that align with observed data. Our approach uses a smoother based on iterating a conditional particle filter with ancestral sampling to first generate plausible trajectories matching observed marginals, and learns the corresponding diffusion model. This approach provides both a generative method for high-quality, smoothed trajectories under complex constraints, and an efficient approximation of the particle smoothing distribution for classical tracking problems. We demonstrate the approach in time-series generation and interpolation tasks, including vehicle tracking and single-cell RNA sequencing data.

Figures (6)

• Figure 1: Trajectories from an SDE with the neural network drift model learned from smoothing trajectories over a zero drift model with the constraints $\pi_0 = \pi_T = \mathcal{N}(0, 1)$, with observations forcing the trajectories to split into two modes.
• Figure 2: In an iteration of the CPF-AS smoother, the reference trajectory (top) is used to create conditional particle filtering trajectories (middle), from which a new reference trajectory is sampled based on weights on the last time step (bottom). The final weights are computed based on a Kernel Density Estimate (KDE) over the distribution $\pi_T = \mathcal{N}(0, 1)$. The trajectories are sampled based on intermediate observations and the reference trajectory, resulting in multiple trajectories in the middle plot following roughly the previous reference.
• Figure 3: (a) Samples from the particle smoothing distribution using the known drift of the double-well system. (b) Trajectories generated from the learned diffusion process. The mean (in blue) and the trajectory samples (in gray) follow similar patterns, and the learned SDE roughly follows the observations without access to them at sampling time.
• Figure 4: Particle smoother marginals (top row) and marginals of the neural SDE model (bottom row), generated from a constrained system where the final distribution is the scikit-learn two-circles data set, and intermediate data consists of $10$ points lying on a circle with a smaller radius (in red). The marginals above are observed at times $t=\{0, 1.4, 1.5, 2.9, T\}$, where $T=3$. Without access to samples from the terminal distribution, the learned model accomplishes modeling a circle with the correct radius, but struggles to separate the two circles from each other.
• Figure 5: The CPF-AS MCMC smoother trajectories cover the known marginals of the single-cell process (projected onto the first two principal axes for visualization on 2D planes), compared to the Iterative Smoothing Bridge which only explores high-density regions of the marginals.