Parameter Inference via Differentiable Diffusion Bridge Importance Sampling

TL;DR

A novel, numerically stable, score matching-based parameter inference framework is presented and demonstrated on biological two- and three-dimensional morphometry data, illustrating its applicability for obtaining insights into the evolution of and relationships between species, including ancestral state reconstruction.

Abstract

We introduce a methodology for performing parameter inference in high-dimensional, non-linear diffusion processes. We illustrate its applicability for obtaining insights into the evolution of and relationships between species, including ancestral state reconstruction. Estimation is performed by utilising score matching to approximate diffusion bridges, which are subsequently used in an importance sampler to estimate log-likelihoods. The entire setup is differentiable, allowing gradient ascent on approximated log-likelihoods. This allows both parameter inference and diffusion mean estimation. This novel, numerically stable, score matching-based parameter inference framework is presented and demonstrated on biological two- and three-dimensional morphometry data.

Figures (7)

• Figure 1: 100 landmarks describing the wing outlines of Papilio ambrax and Papilio slateri, respectively. Data from gbif_m.
• Figure 2: Log-likelihood curves for the conditioned diffusion process evolving landmarks of Papilio ambrax into Papilio slateri. Method 'analytical' is the true log-likelihood, known in this simplified process; 'stable analytical' uses the stable but off-by-a-constant computation presented in Section \ref{['sec:estimation:stable']}; 'simulated' is computed by the simulated likelihood estimation method of 500d2f1b-d6f7-3edc-8f25-66b3ad9b7d1d; 'proposed' uses the stable importance sampler proposed in this work. The latter two methods use 1000 Monte Carlo samples with 1000 simulation time steps. Notice how the off-by-a-constant proposed method exactly captures the shape of the log-likelihood curve, allowing parameter inference.
• Figure 3: Black stars illustrate observations of sampled Brownian motions initiated at the black circle at the origin. Red circle illustrates initial diffusion mean estimate (chosen arbitrarily). The joint log-likelihood of diffusion bridges from the current diffusion mean estimate to each of the observations is computed and the diffusion mean estimate is updated by gradient ascent on it. The red line shows the path taken by the diffusion mean estimate. Red star illustrates final diffusion mean estimate, which coincides almost fully with the true diffusion mean, indicated by the blue triangle.
• Figure 4: Structure of neural network used, shown here corresponding to 100-point two-dimensional landmark shape; the shape is flattened into a 200-dimensional vector and the time point is concatenated. The dark blue skip connections add the values element-wise to the later layers. These skip connections are found to help the model express the score fields properly.
• Figure 5: (a) and (b) illustrate landmark discretisations of the outline of the parietal bone of Canis lupus and Vulpes vulpes specimens, respectively. (c) and (d) show learned Kunita flow diffusion bridges between the landmark configurations of (a) and (b), using two different variance parameter values for the process. Data from boyer2016morphosource.

Theorems & Definitions (8)

• Proposition 2.1: Stochastic Flow of Landmarks
• Theorem 3.1: Doob's $h$-transform
• Theorem 3.2: Time Reversal
• Corollary 3.3: Reverse Time Bridge Process
• Theorem 3.4: Numerically Stable Equivalent Objective Function
• proof
• Proposition 4.1: Importance Sampled Simulated Likelihood Estimation
• proof