Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shared active subspace for multivariate vector-valued functions

Khadija Musayeva, Mickael Binois

TL;DR

This work addresses the problem of obtaining a shared active subspace for vector-valued functions without distributional assumptions. It introduces two broad baselines that reduce vector-valued problems to the scalar setting: gradient-based approaches (linear projection of gradients and convex hull of gradients) and SPD-based approaches (sum of SPD matrices, joint diagonalization, and stepwise eigenvector estimation). Through extensive experiments on a synthetic problem and four real-world inspired optimization tasks under normal and uniform distributions, SPD-based methods, particularly the sum of SPD matrices, demonstrate strong performance and stability, closely matching distribution-specific vector-valued approaches when data are Gaussian. The results highlight the practical value of a distribution-agnostic, SPD-centric framework for identifying shared active subspaces, and the authors provide an accompanying R package SharedAS for practitioners.

Abstract

This paper proposes several approaches as baselines to compute a shared active subspace for multivariate vector-valued functions. The goal is to minimize the deviation between the function evaluations on the original space and those on the reconstructed one. This is done either by manipulating the gradients or the symmetric positive (semi-)definite (SPD) matrices computed from the gradients of each component function so as to get a single structure common to all component functions. These approaches can be applied to any data irrespective of the underlying distribution unlike the existing vector-valued approach that is constrained to a normal distribution. We test the effectiveness of these methods on five optimization problems. The experiments show that, in general, the SPD-level methods are superior to the gradient-level ones, and are close to the vector-valued approach in the case of a normal distribution. Interestingly, in most cases it suffices to take the sum of the SPD matrices to identify the best shared active subspace.

Shared active subspace for multivariate vector-valued functions

TL;DR

This work addresses the problem of obtaining a shared active subspace for vector-valued functions without distributional assumptions. It introduces two broad baselines that reduce vector-valued problems to the scalar setting: gradient-based approaches (linear projection of gradients and convex hull of gradients) and SPD-based approaches (sum of SPD matrices, joint diagonalization, and stepwise eigenvector estimation). Through extensive experiments on a synthetic problem and four real-world inspired optimization tasks under normal and uniform distributions, SPD-based methods, particularly the sum of SPD matrices, demonstrate strong performance and stability, closely matching distribution-specific vector-valued approaches when data are Gaussian. The results highlight the practical value of a distribution-agnostic, SPD-centric framework for identifying shared active subspaces, and the authors provide an accompanying R package SharedAS for practitioners.

Abstract

This paper proposes several approaches as baselines to compute a shared active subspace for multivariate vector-valued functions. The goal is to minimize the deviation between the function evaluations on the original space and those on the reconstructed one. This is done either by manipulating the gradients or the symmetric positive (semi-)definite (SPD) matrices computed from the gradients of each component function so as to get a single structure common to all component functions. These approaches can be applied to any data irrespective of the underlying distribution unlike the existing vector-valued approach that is constrained to a normal distribution. We test the effectiveness of these methods on five optimization problems. The experiments show that, in general, the SPD-level methods are superior to the gradient-level ones, and are close to the vector-valued approach in the case of a normal distribution. Interestingly, in most cases it suffices to take the sum of the SPD matrices to identify the best shared active subspace.
Paper Structure (25 sections, 13 equations, 5 figures, 3 tables)

This paper contains 25 sections, 13 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodological background
  3. Active subspace
  4. Vector-valued active subspace
  5. New baselines to vector-valued active subspace
  6. Manipulation at the level of gradients
  7. Linear projection of gradients
  8. Convex hull of gradients
  9. Manipulation at the level SPD matrices
  10. Sum of SPD matrices
  11. Joint diagonalization of SPD matrices
  12. Stepwise estimation of eigenvectors
  13. Numerical Experiments
  14. Setup
  15. Test problems
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Comparison of the proposed approaches and the method of Zahm based on data sampled from the standard normal distribution. The experiments were repeated $10$ times with the sample size $n=1000$ in each of them.
  • Figure 2: Comparison of the proposed approaches based on data sampled from the uniform distribution. The experiments were repeated $10$ times with the sample size $n=1000$ in each of them.
  • Figure 3: Sufficient summary plots of a subset of methods for the synthetic problem applied to uniformly distributed data. AS1 stands for the active subspace of dimension $1$, and $f_1, f_2$ are the two objectives.
  • Figure 4: Summary plots of SSPD, SEE, and AG for the synthetic problem applied to uniformly distributed data. AS1 stands for the active subspace of dimension $1$, and $f_1, f_2$ are the two objectives.
  • Figure 5: Comparison of the zonoid Mosler13 and potential depths function on a test problem, in the original output space and in copula (rank) space.