Adaptive Nonlinear Data Assimilation through P-Spline Triangular Measure Transport

Abstract

Non-Gaussian statistics are a challenge for data assimilation. Linear methods oversimplify the problem, yet fully nonlinear methods are often too expensive to use in practice. The best solution usually lies between these extremes. Triangular measure transport offers a flexible framework for nonlinear data assimilation. Its success, however, depends on how the map is parametrized. Too much flexibility leads to overfitting; too little misses important structure. To address this balance, we develop an adaptation algorithm that selects a parsimonious parametrization automatically. Our method uses P-spline basis functions and an information criterion as a continuous measure of model complexity. This formulation enables gradient descent and allows efficient, fine-scale adaptation in high-dimensional settings. The resulting algorithm requires no hyperparameter tuning. It adjusts the transport map to the appropriate level of complexity based on the system statistics and ensemble size. We demonstrate its performance in nonlinear, non-Gaussian problems, including a high-dimensional distributed groundwater model.

Figures (10)

• Figure 1: Effect of map complexity in a univariate transport example. A bimodal prior target density (left) is approximated from a finite ensemble. Three monotone map components of increasing flexibility (middle: A, B, C) are fitted. Using the forward map (orange, grey filled line in center row) pushes samples to a Gaussian reference (top row). Insufficient flexibility leaves residual non-Gaussian structure (A, top), while excessive flexibility overfits ensemble fluctuations and yields unstable pushforward $S_\#\pi$ (C, top). An intermediate level of complexity achieves stable Gaussianization (B, top). Likewise, an inverse map (green, grey dashed line in center row) of insufficient flexibility fails to convert a standard Gaussian reference (right) to a non-Gaussian target (A, bottom), while excessive flexibility overfits ensemble fluctuations and yields unstable pullbacks $S^\#\eta$ (C, bottom).
• Figure 2: Assigning monotonously increasing coefficients to each B-spline basis function $B_{i,d}$ (A) results in a monotone function from the superposition $f(x,\boldsymbol{\beta})$ of the scaled basis functions (B). Linearly extrapolating the B-splines furthermore ensures monotonicity beyond the support of the B-spline.
• Figure 3: The effect of the smoothing penalty on the pullback $S^\#\eta$ (top row), the pushforward $S_\#\pi$ (middle row), and the AICc (bottom row). The target distribution $\pi$ is a bivariate wavy distribution. We keep smoothing penalties for the monotone parts of the map component functions $S_1$ and $S_2$ fixed at $\log\lambda=10$ and only adjust the smoothing penalty of the nonmonotone part of $S_2$. The problem is defined artificially difficult with $n=30$ ensemble size and $K=50$ knots to better highlight the influence of the smoothing penalty. For low smoothing penalties, the map tends to overfit, yielding very noisy pushforward and pullback. For high smoothing penalty, the map becomes too simple, only providing very rudimentary transformation. In-between the AIC shows an optimal trade-off between the negative log-likelihood and the effective degrees of freedom.
• Figure 4: (A) Time-averaged ensemble mean RMSEs, averaged across ten random seeds, for EnTFs based on Hermite functions of varying maximum polynomial order ramgraber2023ensemble as well as the adaptive P-spline algorithm proposed in this study. Note that the linear (order 1) EnTF corresponds to an Ensemble Kalman Filter (EnKF). (B) Averaged fraction of effective degrees of freedom to raw degrees of freedom of the adaptive P-spline EnTF for different ensemble sizes.
• Figure 5: Prior (first column) and posterior (second and third column) mean (first row) and marginal standard deviations (second row) for the log-conductivities obtained with both methods. The synthetic truth's log hydraulic conductivity field is illustrated in the top right.