Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Convex Quadratic Optimization with Indicators Over Structured Graphs

Aaresh Bhathena, Salar Fattahi, Andrés Gómez, Simge Küçükyavuz

TL;DR

An exact parametric dynamic programming algorithm whose computational complexity depends explicitly on the treewidth of the Hessian's support graph, its volume growth, and an appropriate margin parameter is developed, which significantly outperforms state-of-the-art solvers.

Abstract

This paper studies convex quadratic minimization problems in which each continuous variable is coupled with a binary indicator variable. We focus on the structured setting where the Hessian matrix of the quadratic term is positive definite and exhibits sparsity. We develop an exact parametric dynamic programming algorithm whose computational complexity depends explicitly on the treewidth of the Hessian's support graph, its volume growth, and an appropriate margin parameter. Under suitable structural conditions, the overall complexity scales linearly with the problem dimension. To demonstrate the practical impact of our approach, we introduce a novel framework for joint forecasting and outlier detection by extending exponential smoothing to time series with outliers. Computational experiments on both synthetic and real data sets show that our method significantly outperforms state-of-the-art solvers.

Solving Convex Quadratic Optimization with Indicators Over Structured Graphs

TL;DR

An exact parametric dynamic programming algorithm whose computational complexity depends explicitly on the treewidth of the Hessian's support graph, its volume growth, and an appropriate margin parameter is developed, which significantly outperforms state-of-the-art solvers.

Abstract

This paper studies convex quadratic minimization problems in which each continuous variable is coupled with a binary indicator variable. We focus on the structured setting where the Hessian matrix of the quadratic term is positive definite and exhibits sparsity. We develop an exact parametric dynamic programming algorithm whose computational complexity depends explicitly on the treewidth of the Hessian's support graph, its volume growth, and an appropriate margin parameter. Under suitable structural conditions, the overall complexity scales linearly with the problem dimension. To demonstrate the practical impact of our approach, we introduce a novel framework for joint forecasting and outlier detection by extending exponential smoothing to time series with outliers. Computational experiments on both synthetic and real data sets show that our method significantly outperforms state-of-the-art solvers.
Paper Structure (45 sections, 13 theorems, 86 equations, 11 figures, 5 tables, 4 algorithms)

This paper contains 45 sections, 13 theorems, 86 equations, 11 figures, 5 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. An overview of our contributions
  3. Related work
  4. Methods based on sequential convex relaxation.
  5. Methods for structured $\bm{Q}$.
  6. Methods based on small treewidth.
  7. Outline and overview
  8. Preliminaries and background
  9. Tree decomposition and treewidth
  10. The local parametric cost
  11. Volume growth of a graph
  12. Application: Exponential smoothing with outlier correction
  13. Exponential smoothing
  14. Exponential smoothing with outlier correction
  15. Dynamic programming via local parametric costs
  16. ...and 30 more sections

Key Result

Lemma 1

Fix any $u\in\{1,2,\ldots,n\}$. The local parametric cost $f_u:\mathbb{R}^{\tau+1}\to\mathbb{R}$ can be written as where, for every $\bm{s}\in\{0,1\}^{n_u}$, $p_{u,\bm{s}}(\bm{\alpha})$ is a strongly convex quadratic function. In particular, let $\mathcal{J}_{u,\bm{s}} = \{i\in \mathcal{J}_u\mid \bm{s}_i=1\}$. Then $p_{u,\bm{s}}(\bm{\alpha}_{\mathcal{B}_u})$ is given by

Figures (11)

  • Figure 1: A graph with a tree decomposition that violates Definition \ref{['def: balanced tree decomposition']} (left), and its corresponding balanced tree decomposition (right).
  • Figure 2: The figure on the left illustrates a graph $\textsf{G}$ with nodes labeled arbitrarily. The center panel shows a balanced tree decomposition of $\textsf{G}$, where the bags are labeled according to the topological ordering (labels shown in bold). The panel on the right shows the resulting labeling of the nodes in $\textsf{G}$ obtained by the described procedure.
  • Figure 3: Top row: SES applied to a signal containing anomalies from the Numenta Anomaly Benchmark (NAB), with smoothing parameters $\beta = 0.5$, $0.2$, and $0.05$ (from left to right). Outliers identified by the NAB ground-truth labels are highlighted with circles. Lower values of $\beta$ reduce sensitivity to outliers but induce a noticeable lag relative to the underlying signal. Bottom row: ESOC effectively detects and removes outliers across all values of $\beta$, while preserving the alignment with the true signal and avoiding lag.
  • Figure 4: Hessian support graph (top) and corresponding tree decomposition (bottom) for ESOC model.
  • Figure 5: Runtime comparison between the proposed algorithm and Gurobi on five real-world time series from the NAB dataset, with smoothing parameters. Signal lengths are varied by truncation. For each signal, smoothing parameters are set to $\beta\in\{0.05,0.2,0.5\}$. “×” denotes the mean runtime. Gurobi runs are terminated after one hour, resulting in flat values at larger signal lengths.
  • ...and 6 more figures

Theorems & Definitions (29)

  • Definition 1: Tree decomposition
  • Definition 2: balanced tree decomposition
  • Lemma 1
  • Lemma 2
  • Proposition 1
  • proof
  • Lemma 3
  • Definition 3: Relevant and irrelevant functions
  • Lemma 4
  • Definition 4: $m$-similarity
  • ...and 19 more