Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BLOC: A Global Optimization Framework for Sparse Covariance Estimation with Non-Convex Penalties

Priyam Das, Trambak Banerjee, Prajamitra Bhuyan

Abstract

We introduce BLOC (Black-box Optimization over Correlation matrices), a general framework for sparse covariance estimation with non-convex penalties. BLOC operates on the manifold of correlation matrices and reparameterizes it via an angular Cholesky mapping, transforming the positive-definite, unit-diagonal constraint into an unconstrained search over a Euclidean hyperrectangle. This enables gradient-free global optimization of diverse objectives, including non-differentiable or black-box losses, using a pattern search routine with adaptive coordinate polling, run-wise restarts to escape local minima, and leveraging up to $d(d-1)$ parallel threads when optimizing a $d$-dimensional correlation matrix. The method is penalty-agnostic and ensures that every iterate is a valid correlation matrix, from which covariance estimates are obtained. We establish convergence guarantees, including stationarity, probabilistic escape from poor local minima, and sublinear rates under smooth convex losses. From a statistical perspective, we prove consistency, convergence rates, and sparsistency for penalized correlation estimators under general conditions, extending sparse covariance theory beyond the Gaussian setting. Empirically, BLOC with nonconvex penalties such as SCAD and MCP outperforms leading estimators in both low- and high-dimensional regimes, achieving lower estimation error and improved sparsity recovery. A parallel implementation enhances scalability, and a proteomic network application demonstrates robust, positive-definite sparse covariance estimation.

BLOC: A Global Optimization Framework for Sparse Covariance Estimation with Non-Convex Penalties

Abstract

We introduce BLOC (Black-box Optimization over Correlation matrices), a general framework for sparse covariance estimation with non-convex penalties. BLOC operates on the manifold of correlation matrices and reparameterizes it via an angular Cholesky mapping, transforming the positive-definite, unit-diagonal constraint into an unconstrained search over a Euclidean hyperrectangle. This enables gradient-free global optimization of diverse objectives, including non-differentiable or black-box losses, using a pattern search routine with adaptive coordinate polling, run-wise restarts to escape local minima, and leveraging up to parallel threads when optimizing a -dimensional correlation matrix. The method is penalty-agnostic and ensures that every iterate is a valid correlation matrix, from which covariance estimates are obtained. We establish convergence guarantees, including stationarity, probabilistic escape from poor local minima, and sublinear rates under smooth convex losses. From a statistical perspective, we prove consistency, convergence rates, and sparsistency for penalized correlation estimators under general conditions, extending sparse covariance theory beyond the Gaussian setting. Empirically, BLOC with nonconvex penalties such as SCAD and MCP outperforms leading estimators in both low- and high-dimensional regimes, achieving lower estimation error and improved sparsity recovery. A parallel implementation enhances scalability, and a proteomic network application demonstrates robust, positive-definite sparse covariance estimation.

Paper Structure

This paper contains 45 sections, 8 theorems, 77 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Penalized Correlation Estimation
  3. Statistical model and correlation-based formulation
  4. General penalized estimation framework
  5. Notation and asymptotic conventions
  6. BLOC
  7. Correlation-matrix reparameterization
  8. Transformation to an unconstrained parameter space
  9. Optimization strategy
  10. BLOC algorithm
  11. Practical considerations
  12. Theory
  13. Statistical guarantees: rates and sparsistency
  14. Theoretical properties of BLOC
  15. Numerical results
  16. ...and 30 more sections

Key Result

Proposition 1

A matrix $\bm{C} \in \mathcal{C}_d$ if and only if there exists a unique lower triangular matrix $\bm{L}$ such that $\bm{C} = \bm{L}\bm{L}^\top,$ where $\bm{L}$ is full rank, has strictly positive diagonal entries, and satisfies $\sum_{j=1}^m l_{mj}^2 = 1$ for each row $m = 1,\dots,d$. $\blacktriang

Figures (3)

  • Figure 1: Illustration of Fermi’s principle and its implementation within BLOC. Top: Classical Fermi’s principle in $\mathbb{R}^N$, where $2N$ coordinate-wise perturbations are generated using a fixed step-size $s$. Middle: BLOC reparameterization, mapping a correlation matrix $\bm{C} \in \mathcal{C}_d$ to an unconstrained angular vector $\boldsymbol\varphi \in \mathbb{R}^N$ via Cholesky and angular representations, with a wrapping map $\mathscr{M}$ enforcing the admissible angular domain $\mathcal{A}_d$. Bottom: Fermi’s principle applied in the angular space, where coordinate-wise perturbations in $\mathbb{R}^N$ are wrapped into $\mathcal{A}_d$ and mapped back to valid correlation matrices.
  • Figure 2: Flowchart of the BLOC algorithm. The method alternates between coordinate-wise pattern searches and adaptive step-size updates across multiple runs, evaluating up to $2N$ candidate perturbations per iteration (with $N=d(d-1)/2$) in parallel, and mapping the final angular solution back to a valid correlation matrix.
  • Figure 3: Estimated sparse correlation heatmaps for five pan-gynecologic cancers using BLOC with SCAD and a pathway-based penalty cover. Within-pathway coherence is preserved, while across-pathway edges highlight tumor-specific differences in integration.

Theorems & Definitions (16)

  • Proposition 1: Correlation matrix bijection
  • Proposition 2: Angular bijection
  • Theorem 1: Convergence Rates
  • Theorem 2: Sparsistency
  • Theorem 3: Stationarity under Coordinatewise Descent Failure
  • Theorem 4: Open-ball reachability under grid-supported refining restarts
  • Theorem 5: Global Convergence in Probability of BLOC
  • Theorem 6: Sublinear Convergence Rate of BLOC
  • Remark 1
  • Remark 2
  • ...and 6 more