Fast Orthogonal Matching Pursuit through Successive Regression
Huiyuan Yu, Jia He, Maggie Cheng
TL;DR
The paper tackles the computational bottlenecks of Orthogonal Matching Pursuit in sparse recovery by introducing OMP-SR, a fast successive-regression variant that preserves the same atom selection as OMP but avoids costly LS inversions via a backtracking coefficient update. It extends the idea to blocks with Blocked Successive Regression (BSR), reducing iterations and enabling faster recovery comparable to or superior to gOMP while retaining recovery guarantees. The authors provide both strong and weak exact-recovery conditions, connecting these guarantees to dictionary coherence and cumulative coherence measures, and show that under conditions like $\mu(2k-1) < 1$ or $\mu_1(l)+\mu_1(n) < 1$, BSR achieves recovery in $\lceil k/c \rceil$ iterations. Experiments on real and synthetic data demonstrate significant speedups with comparable accuracy under noise, highlighting the practical utility of the proposed fast OMP-based algorithms for large-scale sparse representation problems.
Abstract
Orthogonal Matching Pursuit (OMP) has been a powerful method in sparse signal recovery and approximation. However, OMP suffers computational issues when the signal has a large number of non-zeros. This paper advances OMP and its extension called generalized OMP (gOMP) by offering fast algorithms for the orthogonal projection of the input signal at each iteration. The proposed modifications directly reduce the computational complexity of OMP and gOMP. Experiment results verified the improvement in computation time. This paper also provides sufficient conditions for exact signal recovery. For general signals with additive noise, the approximation error is at the same order as OMP (gOMP), but is obtained within much less time.