Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convex relaxation for the generalized maximum-entropy sampling problem

Gabriel Ponte, Marcia Fampa, Jon Lee

TL;DR

This work addresses generalized maximum-entropy sampling (GMESP), which subsumes maximum-entropy sampling (MESP) and binary D-optimality (D-Opt). The authors introduce a convex-optimization–based relaxation, the generalized factorization bound DGFact, built from a factorization C=FF^T and a nonconvex relaxation, together with a convex relaxation DDGFact derived via duality; both bounds connect to and improve upon existing spectral bounds, particularly when the difference s−t is small. They establish key invariance properties, exactness for t=s, and a dual framework with a closed-form dual for certain subproblems, enabling effective variable fixing in branch-and-bound. Computational experiments on a 63-variable covariance matrix show that the generalized factorization bound provides strong pruning and practical solvability for GMESP and CGMESP when s−t is small, with constrained instances benefiting notably from side constraints. The findings offer a scalable, bound-driven approach to GMESP that unifies prior bounds and supports PCA-inspired design problems, while outlining directions for stronger bounds in harder regimes.

Abstract

The generalized maximum-entropy sampling problem (GMESP) is to select an order-$s$ principal submatrix from an order-$n$ covariance matrix, to maximize the product of its $t$ greatest eigenvalues, $0<t\leq s <n$. Introduced more than 25 years ago, GMESP is a natural generalization of two fundamental problems in statistical design theory: (i) maximum-entropy sampling problem (MESP); (ii) binary D-optimality (D-Opt). In the general case, it can be motivated by a selection problem in the context of principal component analysis (PCA). We introduce the first convex-optimization based relaxation for GMESP, study its behavior, compare it to an earlier spectral bound, and demonstrate its use in a branch-and-bound scheme. We find that such an approach is practical when $s-t$ is very small.

Convex relaxation for the generalized maximum-entropy sampling problem

TL;DR

This work addresses generalized maximum-entropy sampling (GMESP), which subsumes maximum-entropy sampling (MESP) and binary D-optimality (D-Opt). The authors introduce a convex-optimization–based relaxation, the generalized factorization bound DGFact, built from a factorization C=FF^T and a nonconvex relaxation, together with a convex relaxation DDGFact derived via duality; both bounds connect to and improve upon existing spectral bounds, particularly when the difference s−t is small. They establish key invariance properties, exactness for t=s, and a dual framework with a closed-form dual for certain subproblems, enabling effective variable fixing in branch-and-bound. Computational experiments on a 63-variable covariance matrix show that the generalized factorization bound provides strong pruning and practical solvability for GMESP and CGMESP when s−t is small, with constrained instances benefiting notably from side constraints. The findings offer a scalable, bound-driven approach to GMESP that unifies prior bounds and supports PCA-inspired design problems, while outlining directions for stronger bounds in harder regimes.

Abstract

The generalized maximum-entropy sampling problem (GMESP) is to select an order- principal submatrix from an order- covariance matrix, to maximize the product of its greatest eigenvalues, . Introduced more than 25 years ago, GMESP is a natural generalization of two fundamental problems in statistical design theory: (i) maximum-entropy sampling problem (MESP); (ii) binary D-optimality (D-Opt). In the general case, it can be motivated by a selection problem in the context of principal component analysis (PCA). We introduce the first convex-optimization based relaxation for GMESP, study its behavior, compare it to an earlier spectral bound, and demonstrate its use in a branch-and-bound scheme. We find that such an approach is practical when is very small.
Paper Structure (8 sections, 18 theorems, 71 equations, 3 figures, 2 tables)

This paper contains 8 sections, 18 theorems, 71 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Generalized factorization bound
  3. Duality for \ref{['DGFact']}
  4. Experiments
  5. Lower bounds
  6. Behavior of upper bounds
  7. Branch-and-bound
  8. Outlook

Key Result

Theorem 1

Figures (3)

  • Figure 1: Gaps for \ref{['GMESP']}, varying $t=s-\kappa$ ($n=63$)
  • Figure 2: Gaps for \ref{['CGMESP']}, varying $t=s-\kappa$ ($n=63$, $m=10$)
  • Figure 3: Gaps for \ref{['GMESP']} and \ref{['CGMESP']}, varying $t$, with $s$ fixed ($n=63$)

Theorems & Definitions (32)

  • Theorem 1
  • proof
  • Lemma 2
  • Theorem 3
  • proof
  • Theorem 4
  • proof
  • Theorem 5
  • Theorem 6
  • Theorem 7
  • ...and 22 more