Matrix Pre-orthogonal Matching Pursuit and Pseudo-Inverse

TL;DR

The paper addresses solving the matrix LS problem $XW=Y$ and obtaining minimum-norm (Moore–Penrose) solutions via pseudo-inverse, extending the pre-orthogonal AFD framework to the matrix setting. It introduces Matrix-POAFD, an iterative method that extracts informative features (columns of $X$) in order of contribution to $Y$ by successive orthogonal projections, yielding a sparse, fast LS solution, and shows a two-step and a one-step approach to compute the pseudo-inverse within the ${\mathcal{H}}-H_K$ formulation. The ${\mathcal{H}}-H_K$ formulation provides conditions for the existence of LS and pseudo-inverse solutions and recasts the problem in RKHS terms, with an induced kernel $K$ linking $L$ to the RKHS $H_K$. Numerical comparisons against standard LS solvers demonstrate that Matrix-POAFD offers competitive or superior accuracy with stable computation times, validating its practicality for large-scale LS and PI problems.

Abstract

We introduce a new fundamental algorithm called Matrix-POAFD to solve the matrix least square problem. The method is based on the matching pursuit principle. The method directly extracts, among the given features as column vectors of the measurement matrix, in the order of their importance, the decisive features for the observing vector. With competitive computational efficiency to the existing sophisticated least square solutions the proposed method, due to its explicit and iterative algorithm process, has the advantage of trading off minimum norms with tolerable error scales. The method inherits recently developed studies in functional space contexts. The second main contribution, also in the algorithm aspect, is to present a two-step iterative computation method for pseudo-inverse. We show that consecutively performing two least square solutions, of which one is to $X$ and the other to $X^*,$ results in the minimum norm least square solution. The two-step algorithm can also be combined into one solving a single least square problem but with respect to $XX^\ast.$ The result is extended to the functional formulation as well. To better explain the idea, as well as for the self-containing purpose, we give short surveys with proofs of key results on closely relevant subjects, including solutions with reproducing kernel Hilbert space setting, AFD type sparse representation in terms of matching pursuit, the general ${\mathcal H}$-$H_K$ formulation and pseudo-inverse of bounded linear operator in Hilbert spaces.

Figures (8)

• Figure 1: Comparison of Error and Computation Time Across Four Methods on a $100 \times 10$ Matrix with Increasing Number of Features
• Figure 2: Comparison of Errors and Computation Times Across Four Methods on a $1000 \times 10$ Matrix with Increasing Number of Features
• Figure : $100 \times 10$ Matrix
• Figure : $3000 \times 30$ Matrix
• Figure : $100 \times 10$ Matrix

Theorems & Definitions (15)

• Theorem 3.1
• Remark 3.2
• Remark 3.3
• Proposition 4.1
• Proposition 4.2
• Theorem 4.3
• Remark 4.4
• Remark 4.5
• Proposition 4.6
• Definition 4.7