Solving Implicit Inverse Problems with Homotopy-Based Regularization Path

TL;DR

Implicit inverse problems require estimating a nonlinear functional of a quantity from noisy observations and are often ill-posed with non-unique solutions. The authors introduce a homotopy-based regularization path that couples an inner adjoint-based gradient optimization for fixed $α$ with an outer continuation that decreases $α$ to trace a stability-enforcing solution path, while enforcing sparsity in the parameter vector $m$. They apply the method to latent dynamic discovery by modeling the dynamics as a linear combination of basis functions $Φ(u)$ and promoting sparsity via a smooth surrogate $g(m)$ together with a hard-thresholding step, with results showing robust recovery under noise and evidence of semi-convergence. The work offers a general framework for solving implicit inverse problems in dynamical systems and discusses practical considerations such as basis selection and opportunities for automatic differentiation and extension to PDEs.

Abstract

Implicit inverse problems, in which noisy observations of a physical quantity are used to infer a nonlinear functional applied to an associated function, are inherently ill posed and often exhibit non uniqueness of solutions. Such problems arise in a range of domains, including the identification of systems governed by Ordinary and Partial Differential Equations (ODEs/PDEs), optimal control, and data assimilation. Their solution is complicated by the nonlinear nature of the underlying constraints and the instability introduced by noise. In this paper, we propose a homotopy based optimization method for solving such problems. Beginning with a regularized constrained formulation that includes a sparsity promoting regularization term, we employ a gradient based algorithm in which gradients with respect to the model parameters are efficiently computed using the adjoint state method. Nonlinear constraints are handled through a Newton Raphson procedure. By solving a sequence of problems with decreasing regularization, we trace a solution path that improves stability and enables the exploration of multiple candidate solutions. The method is applied to the latent dynamics discovery problem in simulation, highlighting performance as a function of ground truth sparsity and semi convergence behavior.

Figures (4)

• Figure 1: $\textbf{m}_1$ (first row), $\textbf{m}_2$ (second row) synthetic data (light blue line) and noisy discrete synthetic data (blue dots) with different amount of noise: from left to right $\boldsymbol{\sigma} = [0.01, 0.1, 0.2]$. $x-$axis shows time. $y-$axis shows the values of the state variable.
• Figure 2: Violin plots of relative errors on parameters and solutions computed with best regularization parameter $\alpha_q^*$ for each trial $q=1,...,n$. First row shows results about parameters $\textbf{m}$. Second row shows results about solutions $\textbf{u}$. $x$-axis reports different levels of noise $\boldsymbol{\sigma} = [0.01, 0.1, 0.2]$. $y$-axis reports the relative error.
• Figure 3: Regularization paths with level of noise $\sigma = 0.1$. First row shows results for $\textbf{m}_1$\ref{['b1-m1']}, second row shows results for $\textbf{m}_2$\ref{['b1-m2']}. The first column reports the regularization paths of the solutions. The colors of the curves transition from blue ($l=0$) to red ($l=99$). The $x-$axis reports the time, the $y-$axis the value of the state variable. The second column reports the regularization paths of the parameters. $x-$axis shows the regularization parameter, $y-$axis shows the parameter values. The ground truth values are symbolized by a cross at the last regularization parameter; the curves and crosses of the same color correspond to the same parameter. The third column shows the regularization path of the relative error on parameters. $x-$axis shows the regularization parameter, $y-$axis shows the relative error.
• Figure 4: Regularization paths with level of noise $\sigma = 0.2$ for $\textbf{m}_1$\ref{['b1-m1']}. Top left: regularization path of the relative error on parameters. $x-$axis shows the regularization parameter, $y-$axis shows the relative error. Top right: regularization paths of the parameters. $x-$axis shows the regularization parameter, $y-$axis shows the parameter values. The ground truth values are symbolized by a cross at the last regularization parameter; the curves and crosses of the same color correspond to the same parameter. Bottom left: terminal part of regularization path of the data loss function \ref{['data loss']}. Bottom right: regularization paths of the solutions. The colors of the curves transition from blue ($l=0$) to red ($l=99$). The $x-$axis reports the time, the $y-$axis the value of the state variable.