Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inverting the Fisher information operator in non-linear models

Dimitri Konen

TL;DR

This work develops an information-geometric theory for nonlinear regression where the regression maps live on a nonlinear submanifold of $L^2_\lambda$, showing that injectivity of the forward linearization $\mathbb I_{\theta_0}$ yields invertibility of the Fisher information operator on carefully constructed spaces $\mathcal H$ and $\mathcal S$. It provides an operational construction of these spaces, proves that $\mathbb I_{\theta_0}^*\mathcal I_\varepsilon \mathbb I_{\theta_0}$ is a linear homeomorphism between them, and describes the associated limiting Gaussian process with RKHS $\mathcal H$, enabling explicit asymptotic minimax lower bounds for differentiable functionals. These results are then instantiated in data-assimilation settings for nonlinear parabolic PDEs, notably reaction-diffusion and Navier–Stokes equations, where the two-sided LAN estimates, inverse-information description, and minimax bounds yield insights into the achievable accuracy for estimating initial states and evolving fields under noisy observations. The framework highlights how Bayesian procedures can attain information-theoretic optimality and provides practical criteria to identify the appropriate hypothesis spaces for inference in infinite-dimensional nonlinear models.

Abstract

We consider non-linear regression models corrupted by generic noise when the regression functions form a non-linear subspace of L^2, relevant in non-linear PDE inverse problems and data assimilation. We show that when the score of the model is injective, the Fisher information operator is automatically invertible between well-identified Hilbert spaces, and we provide an operational characterization of these spaces. This allows us to construct in broad generality the efficient Gaussian involved in the classical minimax and convolution theorems to establish information lower bounds, that are typically achieved by Bayesian algorithms thus showing optimality of these methods. We illustrate our results on time-evolution PDE models for reaction-diffusion and Navier-Stokes equations.

Inverting the Fisher information operator in non-linear models

TL;DR

This work develops an information-geometric theory for nonlinear regression where the regression maps live on a nonlinear submanifold of , showing that injectivity of the forward linearization yields invertibility of the Fisher information operator on carefully constructed spaces and . It provides an operational construction of these spaces, proves that is a linear homeomorphism between them, and describes the associated limiting Gaussian process with RKHS , enabling explicit asymptotic minimax lower bounds for differentiable functionals. These results are then instantiated in data-assimilation settings for nonlinear parabolic PDEs, notably reaction-diffusion and Navier–Stokes equations, where the two-sided LAN estimates, inverse-information description, and minimax bounds yield insights into the achievable accuracy for estimating initial states and evolving fields under noisy observations. The framework highlights how Bayesian procedures can attain information-theoretic optimality and provides practical criteria to identify the appropriate hypothesis spaces for inference in infinite-dimensional nonlinear models.

Abstract

We consider non-linear regression models corrupted by generic noise when the regression functions form a non-linear subspace of L^2, relevant in non-linear PDE inverse problems and data assimilation. We show that when the score of the model is injective, the Fisher information operator is automatically invertible between well-identified Hilbert spaces, and we provide an operational characterization of these spaces. This allows us to construct in broad generality the efficient Gaussian involved in the classical minimax and convolution theorems to establish information lower bounds, that are typically achieved by Bayesian algorithms thus showing optimality of these methods. We illustrate our results on time-evolution PDE models for reaction-diffusion and Navier-Stokes equations.
Paper Structure (29 sections, 22 theorems, 221 equations)

This paper contains 29 sections, 22 theorems, 221 equations.

Table of Contents

  1. Introduction
  2. Information geometry of non-linear models
  3. Main contributions
  4. Data assimilation in non-linear dynamical systems
  5. LAN expansion
  6. Quadratic mean differentiability
  7. LAN expansion of the model
  8. Inverting the Fisher information operator
  9. Extension to the natural domain
  10. Characterization of the range and surjectivity
  11. Injectivity and invertibility
  12. Duality
  13. The optimal limiting Gaussian process
  14. Covariance and series expansion
  15. Support of the process
  16. ...and 14 more sections

Key Result

Proposition 1.2

Assume that Condition CondIntroLinModel holds and that the density $q_\varepsilon$ of the common law of the errors $\varepsilon_i$ in (model) belongs to the Sobolev space $H^1(\mathbb{R}^p)$. Then, the following holds.

Theorems & Definitions (53)

  • Proposition 1.2
  • Theorem 1.4
  • Proposition 2.1
  • proof : Proof of Proposition \ref{['PropQMD']}
  • Proposition 2.2
  • Proposition 2.3
  • proof : Proof of Proposition \ref{['PropInvertIepsilon']}
  • Example 3.1
  • Remark 3.2
  • Example 3.3
  • ...and 43 more