Convergence Analysis of Greedy Algorithms with Adaptive Relaxation in Hilbert Spaces

TL;DR

This work analyzes convergence behavior of greedy approximation schemes in Hilbert spaces with dictionaries. It proves that the Power-Relaxed Greedy Algorithm (PRGA) with step $1/m^{\alpha}$ fails to converge universally when $\alpha>1$, constructing a simple two-dimensional counterexample and establishing a positive lower bound on the residual. To address this, the Convex-Relaxed Greedy Algorithm (CRGA) with an exact line search is proposed, and it is shown to converge with the near-optimal $\mathcal{O}(m^{-1/2})$ rate, matching known lower bounds for greedy-type methods. The results clarify the limitations of power-relaxed schemes and offer a robust alternative with provable convergence guarantees in $A_1(\mathcal{D})$.

Abstract

The Power-Relaxed Greedy Algorithm (PRGA) was introduced as a generalization of the so called Relaxed Greedy Algorithm, introduced by DeVore and Temlyakov, by replacing the relaxation parameter $1/m$ with $1/m^α$, with the aim of improving convergence rates. While the case $α\le 1$ is well understood, the behavior of the algorithm for $α>1$ remained an open problem. In this work, we answer this question and, moreover, we introduce a relaxed greedy algorithm with an optimal step size chosen by exact line search at each iteration.

Theorems & Definitions (14)

• Theorem 2.1
• Theorem 2.2
• Theorem 2.4
• Theorem 2.5
• Lemma 3.1
• proof
• proof : Proof of Theorem \ref{['main']}
• Lemma 5.1: Optimal choice of the relaxation parameter
• proof
• Lemma 5.2