Smoothness and other hyperparameter estimation for inverse problems related to data assimilation

Abstract

We consider Bayesian inverse problems arising in data assimilation for dynamical systems governed by partial and stochastic partial differential equations. The space-time dependent field is inferred jointly with static parameters of the prior and likelihood densities. Particular emphasis is placed on the hyperparameter controlling the prior smoothness and regularity, which is critical in ensuring well-posedness, shaping posterior structure, and determining predictive uncertainty. Commonly it is assumed to be known and fixed a priori; however in this paper we will adopt a hierarchical Bayesian framework in which smoothness and other hyperparameters are treated as unknown and assigned hyperpriors. Posterior inference is performed using Metropolis-within-Gibbs sampling suitable to high dimensions, for which hyperparameter estimation involves little computational overhead. The methodology is demonstrated on inverse problems for the Navier-Stokes equations and the stochastic advection-diffusion equation, under sparse and dense observation regimes, using Gaussian priors with different covariance structure. Numerical results show that jointly estimating the smoothness substantially reduces the errors in uncertainty quantification and parameter estimation induced by smoothness misspecification, by achieving performance comparable to scenarios in which the true smoothness is known.

Figures (11)

• Figure 1: (Stationary regime) Trace (left) and density (right) plots of prior parameters. The trace plots show the MCMC estimates (blue) captures the true value (green) in its distribution, with its cumulative average (red) converging to a value close to the truth. Similarly the density plots display the posterior's estimate (blue) peaking around the true value (red) of both parameters. The prior corresponds to the black dashed line. For both parameters, the true value lies in the support of the densities of the estimates.
• Figure 2: (Stationary regime) Vorticity of the initial condition $v_0$ with the truth (far left) and MCMC estimates: knowing the true $\alpha=2.2$ value (centre left), underestimating with $\alpha=1.5$ (centre), overestimating with $\alpha=3$ (centre right) and estimating $\alpha$ using MwG (far right). The crosses indicate the positions where the vector field is observed. We observe similar estimates from the algorithms knowing the correct smoothness and estimating it, albeit giving a smoother estimate.
• Figure 3: (Stationary regime) Heat map of the variance of the vorticity of $v_0$ vs $\alpha$ (top) in the stationary setting, as previously arranged, with: $\alpha=2.2$ value (far left), $\alpha=1.5$ (centre left), $\alpha=3$ (centre right) and estimating $\alpha$ using MwG (far right). Note the scale of the variance of $v_0$'s estimates being inversely proportional to $\alpha$. We also plot the trajectory of the velocity in the 2nd dimension of those estimates at observed (middle) and unobserved (bottom) locations. The vertical dashed line indicates the time of the last observation. The algorithms knowing the true $\alpha$ and estimating it via MwG exhibit similar behaviour and uncertainty quantification.
• Figure 4: (Chaotic regime) Heat map of the variance of the velocity squared of $v_0$ vs $\alpha$ (top) in the chaotic setting, as previously arranged, with: $\alpha=2.2$ value (far left), $\alpha=1.5$ (centre left), $\alpha=3$ (centre right) and estimating $\alpha$ using MwG (far right). The white circles in the top panels indicate the positions where the vector field is observed. We also plot the trajectory of the velocity in the 2nd dimension of those estimates at observed (middle) and unobserved (bottom) locations. The vertical dashed line indicates the time of the last observation. Note the scale of the variance of the prediction estimates being inversely proportional to $\alpha$.
• Figure 5: Trace (left) and density (right) plot of smoothness parameter estimates in case study 2. The dashed line corresponds to the median estimate $\approx1.1$, close the true value generating the data $\alpha=\nu+1=2$.