Generalized Inversion of Nonlinear Operators

TL;DR

This work develops a broad theory for generalized inversion of nonlinear operators, extending Moore–Penrose-like principles beyond linearity via a nonlinear {1,2}-inverse and a BAS-based pseudo-inverse in normed spaces. It provides existence, uniqueness, and continuity insights, and delivers analytic pseudo-inverses for canonical nonlinear operators (e.g., hard/soft thresholding, ReLU) as well as neural-layer and wavelet-thresholding cases. It then extends to endofunctions with Drazin and left-Drazin inverses, showing when inverses can be expressed as forward-polynomial combinations of the operator, analogous to Cayley–Hamilton-type representations. The paper also develops vanishing-polynomial theory for nonlinear operators, including minimal polynomials, and explores their implications for eigenstructure and forward-application-based inverses, offering constructive paths toward efficient inversion algorithms in learning and image processing. Overall, the framework unifies nonlinear inversion, projection theory, and polynomial-operator representations to enable practical generalized inverses with potential algorithmic benefits in data science tasks.

Abstract

Inversion of operators is a fundamental concept in data processing. Inversion of linear operators is well studied, supported by established theory. When an inverse either does not exist or is not unique, generalized inverses are used. Most notable is the Moore-Penrose inverse, widely used in physics, statistics, and various fields of engineering. This work investigates generalized inversion of nonlinear operators. We first address broadly the desired properties of generalized inverses, guided by the Moore-Penrose axioms. We define the notion for general sets, and then a refinement, termed pseudo-inverse, for normed spaces. We present conditions for existence and uniqueness of a pseudo-inverse and establish theoretical results investigating its properties, such as continuity, its value for operator compositions and projection operators, and others. Analytic expressions are given for the pseudo-inverse of some well-known, non-invertible, nonlinear operators, such as hard- or soft-thresholding and ReLU. We analyze a neural layer and discuss relations to wavelet thresholding. Next, the Drazin inverse, and a relaxation, are investigated for operators with equal domain and range. We present scenarios where inversion is expressible as a linear combination of forward applications of the operator. Such scenarios arise for classes of nonlinear operators with vanishing polynomials, similar to the minimal or characteristic polynomials for matrices. Inversion using forward applications may facilitate the development of new efficient algorithms for approximating generalized inversion of complex nonlinear operators.

Figures (3)

• Figure 1: A symmetric depiction of the $\{1,2\}$-inverse scheme, as detailed in Theorem \ref{['thm:properties of new inverse']}. The inverse 'projects' a point $w$ to $W_0=T(V)$ and then maps it to $V_0$ using $T|_{V_0}^{-1}$ (equivalently, ${T}^\ddagger|_{W_0}$), which is a proper bijection from $W_0$ onto $V_0$. In the other direction, $T$ equivalently first 'projects' a point $v$ onto $V_0$, where $T$ is then a proper bijection onto $W_0$. The generalized projections are done by ${T}^\ddagger T$ in $V$ and by $T{T}^\ddagger$ in $W$.
• Figure 2: An example of a discontinuity in a unique pseudo-inverse due to local extrema, with $T(v)=(v-2)^3-(v-2)$. The pseudo-inverse value at $w_1=T(v_1)$ is $v_1$, which of the two sources of $w_1$ has the smaller norm. For any $\epsilon>0$, $T(v_1)+\epsilon$ has a single source that is greater than $v_2$.
• Figure 3: Necessary and sufficient settings for Drazin-invertibility and left-Drazin invertibility. For a Drazin-invertible operator, the operator $T$ becomes bijective after a finite number of iterations (here, $k=4$). The Drazin inverse is then ${T}^\text{D}=(T|_{T^k(V)})^{-(k+1)}T^k$. For a left-Drazin-invertible operator, the setting is looser: the operator becomes injective after $k$ iterations (here, $k=3$). An inverse ${T}^{\text{LD}}$ with parameter $k$ (for $k\geq 1$) may be then defined as $(T|_{T^k(V)})^{-1}$ on $T^{k+1}(V)$ and arbitrarily elsewhere.

Theorems & Definitions (94)

• Definition 1: Moore-Penrose pseudo-inverse
• Lemma 1
• proof
• Theorem 1
• proof
• Definition 2
• Claim 1
• proof
• Definition 3: Pseudo-inverse for normed spaces
• Claim 2