Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Entropy-based convergence rates of greedy algorithms

Yuwen Li, Jonathan Siegel

TL;DR

The entropy-based convergence estimate is sharp and improves upon the classical width-based analysis of reduced basis methods for elliptic model problems and a novel and simple convergence analysis of the classical orthogonal greedy algorithm for nonlinear dictionary approximation using the metric entropy of the symmetric convex hull of the dictionary.

Abstract

We present convergence estimates of two types of greedy algorithms in terms of the metric entropy of underlying compact sets. In the first part, we measure the error of a standard greedy reduced basis method for parametric PDEs by the metric entropy of the solution manifold in Banach spaces. This contrasts with the classical analysis based on the Kolmogorov n-widths and enables us to obtain direct comparisons between the greedy algorithm error and the entropy numbers, where the multiplicative constants are explicit and simple. The entropy-based convergence estimate is sharp and improves upon the classical width-based analysis of reduced basis methods for elliptic model problems. In the second part, we derive a novel and simple convergence analysis of the classical orthogonal greedy algorithm for nonlinear dictionary approximation using the metric entropy of the symmetric convex hull of the dictionary. This also improves upon existing results by giving a direct comparison between the algorithm error and the metric entropy.

Entropy-based convergence rates of greedy algorithms

TL;DR

The entropy-based convergence estimate is sharp and improves upon the classical width-based analysis of reduced basis methods for elliptic model problems and a novel and simple convergence analysis of the classical orthogonal greedy algorithm for nonlinear dictionary approximation using the metric entropy of the symmetric convex hull of the dictionary.

Abstract

We present convergence estimates of two types of greedy algorithms in terms of the metric entropy of underlying compact sets. In the first part, we measure the error of a standard greedy reduced basis method for parametric PDEs by the metric entropy of the solution manifold in Banach spaces. This contrasts with the classical analysis based on the Kolmogorov n-widths and enables us to obtain direct comparisons between the greedy algorithm error and the entropy numbers, where the multiplicative constants are explicit and simple. The entropy-based convergence estimate is sharp and improves upon the classical width-based analysis of reduced basis methods for elliptic model problems. In the second part, we derive a novel and simple convergence analysis of the classical orthogonal greedy algorithm for nonlinear dictionary approximation using the metric entropy of the symmetric convex hull of the dictionary. This also improves upon existing results by giving a direct comparison between the algorithm error and the metric entropy.
Paper Structure (14 sections, 19 theorems, 122 equations, 3 algorithms)

This paper contains 14 sections, 19 theorems, 122 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Greedy Algorithms for RBMs
  3. Greedy Algorithms for Approximating Functions
  4. Convergence of RBMs in Hilbert spaces
  5. Convergence estimates
  6. Examples
  7. Example 1
  8. Example 2
  9. Convergence of RBMs in Banach spaces
  10. Convergence analysis of OGAs
  11. Direct comparison estimate
  12. Best $n$-term approximation
  13. General target functions
  14. Concluding Remarks

Key Result

Lemma 2.1

Let $V_n=\frac{\pi^\frac{n}{2}}{\Gamma(\frac{n}{2}+1)}$ be the volume of an $\ell_2$-unit ball in $\mathbb R^n$. Let $K$ be a compact set in a Hilbert space $X$. Then for any $v_1, \ldots, v_n\in K$, it holds that where $P_0=0$ and $P_k$ is the orthogonal projection onto $\operatorname{span}\{v_1,\ldots,v_k\}.$

Theorems & Definitions (19)

  • Lemma 2.1
  • Theorem 2.1
  • Corollary 2.1
  • Proposition 2.1
  • Lemma 2.2: Carl's inequality
  • Lemma 2.3
  • Theorem 2.2: Theorem 1 from Ref. CohenDeVore2016
  • Lemma 2.4: Proposition 6.2 from Ref. CarlKyreziPajor1999
  • Lemma 3.1
  • Theorem 3.1
  • ...and 9 more