A forward-only scheme for online learning of proposal distributions in particle filters

TL;DR

This paper tackles the challenge of learning proposal distributions for particle filters in online state-space inference. It introduces a forward-only online scheme based on auxiliary Feynman-Kac models with preserved final target to progressively incorporate future information while keeping $Z_T$ fixed. The authors derive global and local optimality notions, and formulate an iterated forward scheme that updates twisting functions $\varphi_t$ to yield improved proposals without full backward passes. Empirical results on a nonlinear one-dimensional model and a real multivariate stochastic volatility dataset demonstrate that the forward approach improves robustness and stabilizes the marginal-likelihood estimator, often at a modest cost to performance relative to backward methods. Overall, the forward scheme offers a practical, online alternative to backward refinement, with strong stability properties across difficult inference scenarios.

Abstract

We introduce a new online approach for constructing proposal distributions in particle filters using a forward scheme. Our method progressively incorporates future observations to refine proposals. This is in contrast to backward-scheme algorithms that require access to the entire dataset, such as the iterated auxiliary particle filters (Guarniero et al., 2017, arXiv:1511.06286) and controlled sequential Monte Carlo (Heng et al., 2020, arXiv:1708.08396) which leverage all future observations through backward recursion. In comparison, our forward scheme achieves a gradual improvement of proposals that converges toward the proposal targeted by these backward methods. We show that backward approaches can be numerically unstable even in simple settings. Our forward method, however, offers significantly greater robustness with only a minor trade-off in performance, measured by the variance of the marginal likelihood estimator. Numerical experiments on both simulated and real data illustrate the enhanced stability of our forward approach.

Figures (4)

• Figure 1: Comparison of the standard deviation of $\hat{Z}_t$ over $64$ runs after $L=4$ training iterations between a forward algorithm (ours) and controlled SMC heng_controlled_2020. Better performance is associated with lower standard deviation. Each point represents a single simulated dataset colored by the mean (over $64$ iterations) of the sum of the relative empirical weight variances of the bootstrap particle filter (BPF). This variance is expected to be lower for an easier dataset.
• Figure 2: Proportion of simulated datasets on which the standard deviation of $\hat{Z}_t$ over $64$ runs is lower than with the bootstrap particle filter (BPF) after each iteration of the training algorithms (from $1$ to $10$). Denoting $\text{sd}^{\text{BPF}}$ the standard deviation of $\hat{Z}_t$ for the BPF, lighter dots show $P(\text{sd}^{\text{alg}} \le 0.1\text{sd}^{\text{BPF}})$, the proportion of simulated datasets for which the standard deviation of $\text{alg}$ is $10$ time lower than the BPF standard deviation (below the main dots), and $P(\text{sd}^{\text{alg}} \le 10\text{sd}^{\text{BPF}})$ (above the main dots).
• Figure 3: Empirical cumulative distribution of the empirical variances of $\log \hat{Z}_T$ for $d=8$, measured every $100$ steps of the PMMH algorithm over $10$ evaluation.
• Figure 4: Empirical cumulative distribution of the empirical variances of $\log \hat{Z}_T$$d=7$ (bottom), measured every $100$ steps of the PMMH algorithm over $10$ evaluation.