Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The augmented NLP bound for maximum-entropy remote sampling

Gabriel Ponte, Marcia Fampa, Jon Lee

TL;DR

This work advances MERSP for Gaussian random vectors by introducing the augmented NLP bound, which strengthens the existing NLP relaxation through a flexible scaling parameter $\psi$ and convexification, and by proposing diagonal scaling as a preprocessing tool. The authors establish theoretical dominance results—augmented NLP bound over the standard NLP bound, and, under certain conditions, the NLP bound's dominance over the spectral bound—along with exact convex relaxations in the singular-covariance regime. They derive a closed-form optimal augmented scaling $\psi^*$ and develop practical strategies for selecting diagonal-scaling vectors, including a local BFGS-based optimization for NLP-Id. Numerical experiments on benchmark and singular instances show substantial tightening of upper bounds and demonstrate the practical impact for improving MERSP calculations in both regular and rank-deficient cases, with clear guidance on when augmentation and scaling are most beneficial.

Abstract

The maximum-entropy remote sampling problem (MERSP) is to select a subset of s random variables from a set of n random variables, so as to maximize the information concerning a set of target random variables that are not directly observable. We assume throughout that the set of all of these random variables follows a joint Gaussian distribution, and that we have the covariance matrix available. Finally, we measure information using Shannon's differential entropy. The main approach for exact solution of moderate-sized instances of MERSP has been branch-and-bound, and so previous work concentrated on upper bounds. Prior to our work, there were two upper-bounding methods for MERSP: the so-called NLP bound and the spectral bound, both introduced 25 years ago. We are able now to establish domination results between these two upper bounds. We propose an ``augmented NLP bound'' based on a subtle convex relaxation. We provide theoretical guarantees, giving sufficient conditions under which the augmented NLP bound strictly dominates the ordinary NLP bound. In addition, the augmented NLP formulation allows us to derive upper bounds for rank-deficient covariance matrices when they satisfy a technical condition. This is in contrast to the earlier work on the ordinary NLP bound that worked with only positive definite covariance matrices. Finally, we introduce a novel and very effective diagonal-scaling technique for MERSP, employing a positive vector of parameters. Numerical experiments on benchmark instances demonstrate the effectiveness of our approaches in advancing the state of the art for calculating upper bounds on MERSP.

The augmented NLP bound for maximum-entropy remote sampling

TL;DR

This work advances MERSP for Gaussian random vectors by introducing the augmented NLP bound, which strengthens the existing NLP relaxation through a flexible scaling parameter and convexification, and by proposing diagonal scaling as a preprocessing tool. The authors establish theoretical dominance results—augmented NLP bound over the standard NLP bound, and, under certain conditions, the NLP bound's dominance over the spectral bound—along with exact convex relaxations in the singular-covariance regime. They derive a closed-form optimal augmented scaling and develop practical strategies for selecting diagonal-scaling vectors, including a local BFGS-based optimization for NLP-Id. Numerical experiments on benchmark and singular instances show substantial tightening of upper bounds and demonstrate the practical impact for improving MERSP calculations in both regular and rank-deficient cases, with clear guidance on when augmentation and scaling are most beneficial.

Abstract

The maximum-entropy remote sampling problem (MERSP) is to select a subset of s random variables from a set of n random variables, so as to maximize the information concerning a set of target random variables that are not directly observable. We assume throughout that the set of all of these random variables follows a joint Gaussian distribution, and that we have the covariance matrix available. Finally, we measure information using Shannon's differential entropy. The main approach for exact solution of moderate-sized instances of MERSP has been branch-and-bound, and so previous work concentrated on upper bounds. Prior to our work, there were two upper-bounding methods for MERSP: the so-called NLP bound and the spectral bound, both introduced 25 years ago. We are able now to establish domination results between these two upper bounds. We propose an ``augmented NLP bound'' based on a subtle convex relaxation. We provide theoretical guarantees, giving sufficient conditions under which the augmented NLP bound strictly dominates the ordinary NLP bound. In addition, the augmented NLP formulation allows us to derive upper bounds for rank-deficient covariance matrices when they satisfy a technical condition. This is in contrast to the earlier work on the ordinary NLP bound that worked with only positive definite covariance matrices. Finally, we introduce a novel and very effective diagonal-scaling technique for MERSP, employing a positive vector of parameters. Numerical experiments on benchmark instances demonstrate the effectiveness of our approaches in advancing the state of the art for calculating upper bounds on MERSP.
Paper Structure (14 sections, 13 theorems, 40 equations, 7 figures)

This paper contains 14 sections, 13 theorems, 40 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Comparison between MERSP bounds
  3. The augmented NLP bound and its properties
  4. The augmented NLP bound
  5. Exact convex relaxation for the singular case
  6. Optimal augmented scaling for \ref{['MERSP']}
  7. Diagonal scaling for \ref{['MERSP']}
  8. Selection of the diagonal-scaling parameter for the NLP bound
  9. Optimizing the diagonal-scaling parameter for NLP-Id
  10. Numerical experiments
  11. Dominance of the NLP bound over the spectral bound
  12. Impact of the proposed scaling procedures on the NLP bound
  13. MERSP instances with singular covariance matrix
  14. Outlook

Key Result

Theorem 1

Assume that $D \succeq C_2 \succeq C_1$ , $p \geq \mathbf{e}$, $\gamma > 0$, and $0<\gamma d_i\leq \exp(p_i - \sqrt{p_i})$ for $i \in N$. Then $f(\cdot)$ is concave on $0 < x \leq \mathbf{e}$.

Figures (7)

  • Figure 1: Condition ($\Delta\leq 0$) of dominance of NLP bound over spectral bound, $n=40$
  • Figure 2: Gaps for the complementary NLP bound, $n=40$, $s=20$
  • Figure 3: Impact of scaling procedures on complementary NLP-Id bound, $n=40$, $s=20$
  • Figure 4: Largest eigenvalue of $C_2- \psi C_1$ for complementary augmented NLP bound, $n=40$
  • Figure 5: Evaluation of augmented NLP bound for rank-deficient covariance matrices, $n\!=\!50,s\!=\!25$
  • ...and 2 more figures

Theorems & Definitions (27)

  • Theorem 1: AFLW_Remote
  • Theorem 2: AFLW_Remote
  • Theorem 3
  • proof
  • Corollary 4
  • proof
  • Remark 5
  • Lemma 6
  • proof
  • Theorem 7
  • ...and 17 more