Optimal experimental design for parameter estimation in the presence of observation noise

TL;DR

The paper addresses parameter identifiability in dynamical systems under observation noise and proposes an information-theoretic experimental design framework that combines a local Fisher information matrix $\mathcal{F}$ and a global Sobol' indices-based matrix $\mathcal{G}$ to select optimal observation times. It applies the approach to a logistic growth model under IID and Ornstein–Uhlenbeck autocorrelated noise, showing that noise structure significantly shifts optimal sampling and the resulting parameter uncertainty. The key finding is that optimized timing reduces uncertainty and can enable accurate parameter estimates with fewer measurements, with the global and local criteria largely agreeing on time-point placement. The framework is general and extensible to other ODE/PDE models and noise models, offering a practical tool for experimental planning in biology and ecology and paving the way for multi-objective extensions and noise-diagnosis methods.

Abstract

Using mathematical models to assist in the interpretation of experiments is becoming increasingly important in research across applied mathematics, and in particular in biology and ecology. In this context, accurate parameter estimation is crucial; model parameters are used to both quantify observed behaviour, characterise behaviours that cannot be directly measured and make quantitative predictions. The extent to which parameter estimates are constrained by the quality and quantity of available data is known as parameter identifiability, and it is widely understood that for many dynamical models the uncertainty in parameter estimates can vary over orders of magnitude as the time points at which data are collected are varied. Here, we use both local sensitivity measures derived from the Fisher Information Matrix and global measures derived from Sobol' indices to explore how parameter uncertainty changes as the number of measurements, and their placement in time, are varied. We use these measures within an optimisation algorithm to determine the observation times that give rise to the lowest uncertainty in parameter estimates. Applying our framework to models in which the observation noise is both correlated and uncorrelated demonstrates that correlations in observation noise can significantly impact the optimal time points for observing a system, and highlights that proper consideration of observation noise should be a crucial part of the experimental design process.

Figures (8)

• Figure 1: Parameter identifiability under the different noise models, with data collected at 11 regularly spaced time points $t=0,8,16,\ldots,80$. The left column shows the sampled data, the underlying model solution using the input parameters, and the solution generated using the MLE parameters. The shaded region shows the 95% prediction interval generated using the 95% confidence intervals from the profile likelihoods. The right-most three panels show the profile likelihoods for each parameter. The results in the top row are generated using uncorrelated Gaussian noise with $\sigma_{\rm{IID}}^2=9.0$. The results in the middle row are generated using correlated OU noise with $\phi=0.02$ and $\sigma_{\rm{OU}}^2/(2\phi)=9.0$. The results in the bottom row are generated using a misspecified noise model: the data are generated using correlated OU noise with $\phi=0.02$ and $\sigma_{\rm{OU}}^2/(2\phi)=9.0$, whilst the analysis is carried out assuming IID Gaussian noise with $\sigma_{\rm{IID}}^2=9.0$.
• Figure 2: Change in the mean $95\%$ confidence interval for parameters $r$, $K$ and $C_0$ as the variance increases under both correlated and uncorrelated noise, and when the noise is misspecified (as per Figure \ref{['fig:posterior_profile']}). Top row: $\sigma^2=\sigma_\text{IID}^2=\sigma_\text{OU}^2/(2\phi)$ with $\phi=0.02$ so that the variance is equivalent across the different noise models. Bottom row: $\sigma^2=\sigma_{\rm{IID}}^2=\sigma_{\rm{OU}}^2$ with $\phi=0.02$ so that the variance is larger in the correlated noise process than the uncorrelated noise process. In all cases, results were generated by averaging the confidence interval width from 1000 simulations for each noise model and parameter set, and 11 observations were used.
• Figure 3: The results of optimising experimental design in the uncorrelated noise case as the number of observations, $n_s$, is varied from three to ten. The top row shows the results from evenly distributed observations, whilst the second row shows the optimal experimental design derived using the Fisher information matrix, and the bottom row shows the optimal experimental design derived using the global information matrix. In each case, the left-hand plots show the observations time points, whilst the right-hand three plots show the profile likelihoods obtained from five observations. In all plots, $\sigma_{\rm{IID}}^2 = 9.0$.
• Figure 4: (a) Fisher information sensitivity and (b) global information sensitivity, formulated in terms of the gradient of the output, $C(t;\bm{\theta})$, with respect to the individual parameters. We evaluate the sensitivity at $t_k=0,2,4,\ldots,80$, with the Fisher information sensitivity calculated as $\theta_i\,\partial C(t_k)/\partial\theta_i$ and the global information sensitivity calculated as $S_i(t_k)$, as defined in Equation \ref{['Sobol_total']}.
• Figure 5: The results of optimising experimental design in the correlated noise case as the number of observations, $n_s$, is varied from three to ten. The top row shows the results from evenly distributed observations, whilst the second row shows the optimal experimental design derived using the Fisher information matrix, and the bottom row shows the optimal experimental design derived using the global information matrix. In each case, the left-hand plots show the observations time points, whilst the right-hand three plots show the profile likelihoods obtained from five observations. In all plots, $\sigma_{\rm{OU}}^2/(2\phi)=9.0$.