Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Surrogate to Poincaré inequalities on manifolds for structured dimension reduction in nonlinear feature spaces

Pasco Alexandre, Nouy Anthony

TL;DR

This work tackles the problem of approximating high-dimensional differentiable functions by composing a nonlinear feature map with a low-dimensional profile: u(X,Y) ≈ f(g(X), Y). It develops a quadratic surrogate L_{X,m}(g) that upper-bounds the non-convex Poincaré-based loss using M(X) and V_m(X), enabling eigenproblem-based minimization when the feature map is linear in a fixed basis. The paper establishes near-optimality results for grouped dimension reduction by linking to tensor decompositions (HOSVD/Tucker) and discusses hierarchical extensions, along with practical limitations related to tensorized sampling. Numerical experiments on a polynomial benchmark show the surrogate can recover low-dimensional structure when it exists and remains competitive otherwise, offering a scalable route to structured nonlinear dimension reduction in high dimensions.

Abstract

This paper is concerned with the approximation of continuously differentiable functions with high-dimensional input by a composition of two functions: a feature map that extracts few features from the input space, and a profile function that approximates the target function taking the features as its low-dimensional input. We focus on the construction of structured nonlinear feature maps, that extract features on separate groups of variables, using a recently introduced gradient-based method that leverages Poincaré inequalities on nonlinear manifolds. This method consists in minimizing a non-convex loss functional, which can be a challenging task, especially for small training samples. We first investigate a collective setting, in which we construct a feature map suitable to a parametrized family of high-dimensional functions. In this setting we introduce a new quadratic surrogate to the non-convex loss function and show an upper bound on the latter. We then investigate a grouped setting, in which we construct separate feature maps for separate groups of inputs, and we show that this setting is almost equivalent to multiple collective settings, one for each group of variables.

Surrogate to Poincaré inequalities on manifolds for structured dimension reduction in nonlinear feature spaces

TL;DR

This work tackles the problem of approximating high-dimensional differentiable functions by composing a nonlinear feature map with a low-dimensional profile: u(X,Y) ≈ f(g(X), Y). It develops a quadratic surrogate L_{X,m}(g) that upper-bounds the non-convex Poincaré-based loss using M(X) and V_m(X), enabling eigenproblem-based minimization when the feature map is linear in a fixed basis. The paper establishes near-optimality results for grouped dimension reduction by linking to tensor decompositions (HOSVD/Tucker) and discusses hierarchical extensions, along with practical limitations related to tensorized sampling. Numerical experiments on a polynomial benchmark show the surrogate can recover low-dimensional structure when it exists and remains competitive otherwise, offering a scalable route to structured nonlinear dimension reduction in high dimensions.

Abstract

This paper is concerned with the approximation of continuously differentiable functions with high-dimensional input by a composition of two functions: a feature map that extracts few features from the input space, and a profile function that approximates the target function taking the features as its low-dimensional input. We focus on the construction of structured nonlinear feature maps, that extract features on separate groups of variables, using a recently introduced gradient-based method that leverages Poincaré inequalities on nonlinear manifolds. This method consists in minimizing a non-convex loss functional, which can be a challenging task, especially for small training samples. We first investigate a collective setting, in which we construct a feature map suitable to a parametrized family of high-dimensional functions. In this setting we introduce a new quadratic surrogate to the non-convex loss function and show an upper bound on the latter. We then investigate a grouped setting, in which we construct separate feature maps for separate groups of inputs, and we show that this setting is almost equivalent to multiple collective settings, one for each group of variables.
Paper Structure (22 sections, 13 theorems, 88 equations, 2 figures)

This paper contains 22 sections, 13 theorems, 88 equations, 2 figures.

Table of Contents

  1. Keywords.
  2. MSC Classification.
  3. Introduction
  4. Contributions and outline
  5. Collective dimension reduction
  6. Truncation of the Poincaré inequality based loss
  7. Quadratic surrogate to the truncated loss
  8. The surrogate as an upper bound
  9. Minimizing the surrogate
  10. Two groups setting
  11. Bilinear regression function
  12. Unconstrained regression function
  13. Multiple groups setting
  14. Multilinear regression function
  15. Unconstrained regression function
  16. ...and 7 more sections

Key Result

Proposition 2.3

Let $\mathcal{J}_{\mathcal{X}}$, $\mathcal{J}_{\mathcal{X}, m}$ and $\varepsilon_m$ be as defined respectively in equ:def of J collective, equ:def J collective truncated and equ:J collective lower bound eigen values. Then for any $g\in\mathcal{G}_m$, Moreover, if $g^*$ is a minimizer of $\mathcal{J}_{\mathcal{X}, m}$ over $\mathcal{G}_m$ then

Figures (2)

  • Figure 1: Evolution of quantiles with respect to the size of the training sample for $u_3$ with $m=3$. The quantiles $50\%$, $90\%$ and $100\%$ are represented respectively by the continuous, dashed and dotted lines.
  • Figure 2: Evolution of quantiles with respect to the size of the training sample for $u_3$ with $m=2$. The quantiles $50\%$, $90\%$ and $100\%$ are represented respectively by the continuous, dashed and dotted lines.

Theorems & Definitions (32)

  • Remark 2.1
  • Remark 2.2
  • Proposition 2.3
  • proof
  • Lemma 2.4
  • proof
  • Lemma 2.5
  • proof
  • Remark 2.6
  • Remark 2.7
  • ...and 22 more