Computing adjoint mismatch of linear maps

TL;DR

This work tackles the challenge of computing the operator norm difference $||A - V||$ when only black-box access to $A$ and to $V^*$ is available and full matrix storage is impractical. It introduces two stochastic, adjoint-free algorithms that iteratively ascend toward the top singular value, provably converging almost surely to $||A - V||$ and yielding corresponding left/right singular vectors. The methods maintain $\mathcal{O}(\max\{m,d\})$ storage and rely on random search directions on tangent spaces, with closed-form step-size rules and convergence analyses that handle singular-value multiplicities under mild assumptions. Numerical experiments highlight faster convergence for the two-step-size variant in higher dimensions, validate adjointness checks in tomography (forward/backprojection), and confirm the predicted convergence behavior of the singular-value/eigenvector relations. The results offer practical tools for adjoint-mismatch detection in imaging applications and a framework for adjoint-free operator-norm estimation in finite-dimensional settings.

Abstract

This paper considers the problem of detecting adjoint mismatch for two linear maps. To clarify, this means that we aim to calculate the operator norm for the difference of two linear maps, where for one we only have a black-box implementation for the evaluation of the map, and for the other we only have a black-box for the evaluation of the adjoint map. We give two stochastic algorithms for which we prove the almost sure convergence to the operator norm. The algorithm is a random search method for a generalization of the Rayleigh quotient and uses optimal step sizes. Additionally, a convergence analysis is done for the corresponding singular vector and the respective eigenvalue equation.

Figures (4)

• Figure 1: Shape of $q_k^2(\tau)$ from \ref{['eq:q_2_function_tau']} for different cases of the signs of $a_kb_k + c_kd_k$ and $a_k^2 - b_k^2 + c_k^2 - d_k^2$.
• Figure 2: Results for 50 runs of Algorithm \ref{['alg:OpNorm2']} (blue) and Algorithm \ref{['alg:OpNorm3']} (orange) for Gaussian matrices of different sizes.
• Figure 3: Results for 50 runs of BLSW24 (blue) and Algorithm \ref{['alg:OpNorm3']} (orange) for Gaussian matrices $A$, where $V = 0$ and $u^k = A v^k / \|A v^k\|$ for BLSW24, of different sizes.
• Figure 4: Results for 50 runs of Algorithm \ref{['alg:OpNorm3']} for Gaussian matrices of different sizes. The minimal error within the $n$-th iteration for the corresponding eigenvalue equation is given.

Theorems & Definitions (70)

• Remark 2.1: Non-negativity of objective
• Remark 2.2: Sampling of $x^k$ and $w^{k}$
• Proposition 2.3
• proof
• Remark 2.4: Monotone and convergent sequence
• Remark 2.5: Algorithm \ref{['alg:OpNorm2']} is a stochastic projected gradient method
• Lemma 2.6
• proof
• Lemma 2.7
• proof