Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Greedy Recombination Interpolation Method (GRIM)

Terry Lyons, Andrew D. McLeod

TL;DR

GRIM introduces a hybrid sparse-approximation framework that merges data-driven greedy growth with thinning via recombination, enabling sparse representations of a dense linear combination of features. By matching the target function $\varphi$ on an adaptively growing subset of data $\Sigma$ and using recombination to minimize support, GRIM achieves provable sparsity bounds and convexity-preserving weights. Theoretical results bound the algorithm’s complexity and guarantee convergence within a finite number of steps, while numerical experiments show GRIM performing favorably against GEIM, LASSO, and kernel-quadrature baselines across cubature-like tasks and kernel-based inference settings. This approach extends the reach of recombination beyond measure-reduction contexts and yields practical sparse surrogates for repeated-inference scenarios.

Abstract

In this paper we develop the Greedy Recombination Interpolation Method (GRIM) for finding sparse approximations of functions initially given as linear combinations of some (large) number of simpler functions. In a similar spirit to the CoSaMP algorithm, GRIM combines dynamic growth-based interpolation techniques and thinning-based reduction techniques. The dynamic growth-based aspect is a modification of the greedy growth utilised in the Generalised Empirical Interpolation Method (GEIM). A consequence of the modification is that our growth is not restricted to being one-per-step as it is in GEIM. The thinning-based aspect is carried out by recombination, which is the crucial component of the recent ground-breaking convex kernel quadrature method. GRIM provides the first use of recombination outside the setting of reducing the support of a measure. The sparsity of the approximation found by GRIM is controlled by the geometric concentration of the data in a sense that is related to a particular packing number of the data. We apply GRIM to a kernel quadrature task for the radial basis function kernel, and verify that its performance matches that of other contemporary kernel quadrature techniques.

Greedy Recombination Interpolation Method (GRIM)

TL;DR

GRIM introduces a hybrid sparse-approximation framework that merges data-driven greedy growth with thinning via recombination, enabling sparse representations of a dense linear combination of features. By matching the target function on an adaptively growing subset of data and using recombination to minimize support, GRIM achieves provable sparsity bounds and convexity-preserving weights. Theoretical results bound the algorithm’s complexity and guarantee convergence within a finite number of steps, while numerical experiments show GRIM performing favorably against GEIM, LASSO, and kernel-quadrature baselines across cubature-like tasks and kernel-based inference settings. This approach extends the reach of recombination beyond measure-reduction contexts and yields practical sparse surrogates for repeated-inference scenarios.

Abstract

In this paper we develop the Greedy Recombination Interpolation Method (GRIM) for finding sparse approximations of functions initially given as linear combinations of some (large) number of simpler functions. In a similar spirit to the CoSaMP algorithm, GRIM combines dynamic growth-based interpolation techniques and thinning-based reduction techniques. The dynamic growth-based aspect is a modification of the greedy growth utilised in the Generalised Empirical Interpolation Method (GEIM). A consequence of the modification is that our growth is not restricted to being one-per-step as it is in GEIM. The thinning-based aspect is carried out by recombination, which is the crucial component of the recent ground-breaking convex kernel quadrature method. GRIM provides the first use of recombination outside the setting of reducing the support of a measure. The sparsity of the approximation found by GRIM is controlled by the geometric concentration of the data in a sense that is related to a particular packing number of the data. We apply GRIM to a kernel quadrature task for the radial basis function kernel, and verify that its performance matches that of other contemporary kernel quadrature techniques.
Paper Structure (11 sections, 9 theorems, 103 equations, 2 figures, 3 tables)

This paper contains 11 sections, 9 theorems, 103 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Mathematical Framework & Motivation
  3. Recombination Thinning
  4. The Banach GRIM Algorithm
  5. Complexity Cost
  6. Convergence Analysis
  7. Notation and Conventions
  8. Main Theoretical Result
  9. Supplementary Lemmata
  10. Proof of Main Theoretical Result
  11. Numerical Examples

Key Result

Lemma 3.1

Assume $X$ is a Banach space with dual space $X^{\ast}$, that ${\cal N} \in {\mathbb Z}_{\geq 1}$, and that $m \in {\mathbb Z}_{\geq 0}$. Define ${\cal M} := \min \{ {\cal N} , m + 1 \}$. Let ${\cal F} := \{ f_1 , \ldots , f_{{\cal N}} \} \subset X$ be a collection of non-zero elements and $L = \{ \ and such that the element $u \in \mathop{\mathrm{Span}}\nolimits({\cal F}) \subset X$ defined by F

Figures (2)

  • Figure 1: 3D Road Network Results - The average $\log( \text{WCE}(u,\varphi,{\cal H}_k)^2 )$ over 20 trials is plotted for each method. The shaded regions show their standard deviations.
  • Figure 2: Combined Cycle Power Plant Results - The average $\log( \text{WCE}(u,\varphi,{\cal H}_k)^2 )$ over 20 trials is plotted for each method. The shaded regions show their standard deviations.

Theorems & Definitions (22)

  • Lemma 3.1: Recombination Thinning
  • proof : Proof of Lemma \ref{['Banach_recombination_lemma']}
  • Lemma 5.1: Banach Extension Step Complexity Cost
  • proof : Proof of Lemma \ref{['banach_ext_step_cost_lemma']}
  • Lemma 5.2
  • proof : Proof of Lemma \ref{['Banach_recomb_step_cost_lemma']}
  • Lemma 5.3: Banach GRIM Complexity Cost
  • proof : Proof of Lemma \ref{['GRIM_cost_lemma']}
  • Lemma 6.1
  • proof : Proof of Lemma \ref{['conv_hull_equiv_form_lemma']}
  • ...and 12 more