Neural Likelihood Approximation for Integer Valued Time Series Data

TL;DR

The paper tackles parameter inference for integer-valued time series where likelihoods are intractable. It introduces a neural conditional density estimator using autoregressive causal CNNs and a discretised mixture of logistics to model $p(y_i|y_{1:i-1},\boldsymbol{\theta})$, trained via Sequential Neural Likelihood on unconditional simulations. The approach yields accurate posterior approximations that rival exact methods like PMMH while offering substantial speedups, particularly for long time series and tall data. Practically, this neural surrogate enables efficient Bayesian inference for stochastic population models in ecology and epidemiology, with potential applicability to other Markov jump processes and multivariate time series.

Abstract

Stochastic processes defined on integer valued state spaces are popular within the physical and biological sciences. These models are necessary for capturing the dynamics of small systems where the individual nature of the populations cannot be ignored and stochastic effects are important. The inference of the parameters of such models, from time series data, is challenging due to intractability of the likelihood. To work at all, current simulation based inference methods require the generation of realisations of the model conditional on the data, which can be both tricky to implement and computationally expensive. In this paper we instead construct a neural likelihood approximation that can be trained using unconditional simulation of the underlying model, which is much simpler. We demonstrate our method by performing inference on a number of ecological and epidemiological models, showing that we can accurately approximate the true posterior while achieving significant computational speed ups compared to current best methods.

Figures (17)

• Figure 1: An example of the dependency structure in a causal convolutional neural network with 4 layers and using a convolution kernel length of 3 in every layer. Notice that paths from $\boldsymbol{o}_i$ back to the input elements $y_j$ only exist for $j \leq i$.
• Figure 2: Evaluation of the causal CNN for one-dimensional inputs. The numbers $(a, b)$ in the causal convolution blocks denote the number of input/output channels for the convolution layer. The quantity $d_h$ denotes the number of channels used in the hidden layers, and $h$ is the number of residual blocks used. We have used $\oplus$ to denote addition of a vector and sequence of vectors broadcasted across the time axis.
• Figure 3: Posterior Pairs Plots for SIR Model Experiments. Left: single outbreak experiment from a population of $N=50$. Right: experiment using 500 households. For the univariate posteriors, we show all 10 runs of SNL, but only one of the runs for the bivariate posterior.
• Figure 4: Comparison of ESS/s for SNL and PMMH for all three models. The lines plotted for SNL are the averages over the runs, and the error bars correspond to the maximum and minimum. All plots are on a log-log scale. The horizontal axis of the left panel scales the observed dataset size, showing that our method has improved scaling compared to the gold standard PMMH. The horizontal axis of the middle panel scales the system size and similarly shows improved scaling. The horizontal axis of the right panel scales the observation accuracy, as discussed in Section \ref{['sec:pp']}. We observe the well-known phenomenon that PMMH runtime degrades with more accurate data, but SNL improves.
• Figure 5: $(R_0, q)$ joint posterior samples for SEIAR experiment with $N=500$.