Surrogate to Poincaré inequalities on manifolds for dimension reduction in nonlinear feature spaces

TL;DR

The work addresses learning nonlinear feature maps $g$ to approximate $u$ by a composition $f\circ g$ through a Poincaré-inequality-based surrogate. It introduces convex surrogates, notably $\mathcal{L}_1$, and extends to multiple features via $\mathcal{L}_{m,j}$ with a greedy learning strategy, providing suboptimality bounds under deviation inequalities. The authors prove that $\mathcal{L}_1$ is a quadratic form in a fixed basis, enabling efficient generalized eigenvalue computations, and show similar structure for the multi-feature surrogates. Numerical experiments on several benchmarks demonstrate that the surrogate-based approaches can outperform direct optimization of $\mathcal{J}$, especially for small training sets and single-feature cases, supporting practical applicability for nonlinear dimension reduction in data-scarce settings.

Abstract

We aim to approximate a continuously differentiable function $u:\mathbb{R}^d \rightarrow \mathbb{R}$ by a composition of functions $f\circ g$ where $g:\mathbb{R}^d \rightarrow \mathbb{R}^m$, $m\leq d$, and $f : \mathbb{R}^m \rightarrow \mathbb{R}$ are built in a two stage procedure. For a fixed $g$, we build $f$ using classical regression methods, involving evaluations of $u$. Recent works proposed to build a nonlinear $g$ by minimizing a loss function $\mathcal{J}(g)$ derived from Poincaré inequalities on manifolds, involving evaluations of the gradient of $u$. A problem is that minimizing $\mathcal{J}$ may be a challenging task. Hence in this work, we introduce new convex surrogates to $\mathcal{J}$. Leveraging concentration inequalities, we provide sub-optimality results for a class of functions $g$, including polynomials, and a wide class of input probability measures. We investigate performances on different benchmarks for various training sample sizes. We show that our approach outperforms standard iterative methods for minimizing the training Poincaré inequality based loss, often resulting in better approximation errors, especially for rather small training sets and $m=1$.

Figures (6)

• Figure 1: Evolution of quantiles with respect to the size of the training sample for $u_1$ with $m=1$. The quantiles $50\%$, $90\%$ and $100\%$ are represented respectively by the continuous, dashed and dotted lines.
• Figure 2: Evolution of quantiles with respect to the size of the training sample for $u_3$ with $m=1$. The quantiles $50\%$, $90\%$ and $100\%$ are represented respectively by the continuous, dashed and dotted lines.
• Figure 3: Evolution of quantiles with respect to the size of the training sample for $u_4$ with $m=1$. The quantiles $50\%$, $90\%$ and $100\%$ are represented respectively by the continuous, dashed and dotted lines.
• Figure 4: Evolution of quantiles with respect to the size of the training sample for $u_2$ with $m=2$. The quantiles $50\%$, $90\%$ and $100\%$ are represented respectively by the continuous, dashed and dotted lines.
• Figure 5: Evolution of quantiles with respect to the size of the training sample for $u_3$ with $m=2$. The quantiles $50\%$, $90\%$ and $100\%$ are represented respectively by the continuous, dashed and dotted lines.

Theorems & Definitions (44)

• Definition 2.1: Poincaré inequality on Riemannian submanifold of $\mathbb{R}^d$
• Proposition 2.1
• Definition 3.1: $s$-concave probability measure
• Proposition 3.1
• Proposition 3.2
• proof
• Proposition 3.3: Small deviations
• proof
• Proposition 3.4: Large deviations
• proof