Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimization of Sums of Bivariate Functions: An Introduction to Relaxation-Based Methods for the Case of Finite Domains

Nils Müller

TL;DR

This work introduces relaxation-based optimization for functions that decompose as a sum of bivariate terms on finite domains, i.e., $F(x)=\sum_{(i,j)\in\mathcal{E}} f_{i,j}(x_i,x_j)$. By replacing the search domain with probability measures and studying tree- and star-structured relaxations, it derives tractable formulations via linear programming, dynamic programming on trees, and entropy-regularized duals, together with dual-to-primal recovery procedures. The paper proves NP-hardness for general graphs while showing exact recoverability and efficient solutions for tree-structured cases, and it develops several algorithms (coordinate descent, dual LP, block-coordinate ascent) that leverage these relaxations. Empirical results on random sums, vertex coloring, and signal reconstruction illustrate when relaxation-based methods excel and where they struggle, highlighting the trade-offs between primal quality and dual convergence across problem classes. Overall, the framework offers a principled pathway to tractable global optimization for sums of bivariates with broad relevance to inverse problems and graphical-model-like settings, and it lays groundwork for future extensions to more general graph structures and regularized objectives.

Abstract

We study the optimization of functions with $n>2$ arguments that have a representation as a sum of several functions that have only $2$ of the $n$ arguments each, termed sums of bivariates, on finite domains. The complexity of optimizing sums of bivariates is shown to be NP-equivalent and it is shown that there exists free lunch in the optimization of sums of bivariates. Based on measure-valued extensions of the objective function, so-called relaxations, $\ell^2$-approximation, and entropy-regularization, we derive several tractable problem formulations solvable with linear programming, coordinate ascent as well as with closed-form solutions. The limits of applying tractable versions of such relaxations to sums of bivariates are investigated using general results for reconstructing measures from their bivariate marginals. Experiments in which the derived algorithms are applied to random functions, vertex coloring, and signal reconstruction problems provide insights into qualitatively different function classes that can be modeled as sums of bivariates.

Optimization of Sums of Bivariate Functions: An Introduction to Relaxation-Based Methods for the Case of Finite Domains

TL;DR

This work introduces relaxation-based optimization for functions that decompose as a sum of bivariate terms on finite domains, i.e., . By replacing the search domain with probability measures and studying tree- and star-structured relaxations, it derives tractable formulations via linear programming, dynamic programming on trees, and entropy-regularized duals, together with dual-to-primal recovery procedures. The paper proves NP-hardness for general graphs while showing exact recoverability and efficient solutions for tree-structured cases, and it develops several algorithms (coordinate descent, dual LP, block-coordinate ascent) that leverage these relaxations. Empirical results on random sums, vertex coloring, and signal reconstruction illustrate when relaxation-based methods excel and where they struggle, highlighting the trade-offs between primal quality and dual convergence across problem classes. Overall, the framework offers a principled pathway to tractable global optimization for sums of bivariates with broad relevance to inverse problems and graphical-model-like settings, and it lays groundwork for future extensions to more general graph structures and regularized objectives.

Abstract

We study the optimization of functions with arguments that have a representation as a sum of several functions that have only of the arguments each, termed sums of bivariates, on finite domains. The complexity of optimizing sums of bivariates is shown to be NP-equivalent and it is shown that there exists free lunch in the optimization of sums of bivariates. Based on measure-valued extensions of the objective function, so-called relaxations, -approximation, and entropy-regularization, we derive several tractable problem formulations solvable with linear programming, coordinate ascent as well as with closed-form solutions. The limits of applying tractable versions of such relaxations to sums of bivariates are investigated using general results for reconstructing measures from their bivariate marginals. Experiments in which the derived algorithms are applied to random functions, vertex coloring, and signal reconstruction problems provide insights into qualitatively different function classes that can be modeled as sums of bivariates.

Paper Structure

This paper contains 28 sections, 19 theorems, 82 equations, 7 figures, 6 algorithms.

Table of Contents

  1. Introduction
  2. Motivation.
  3. Related Work.
  4. Outline.
  5. Acknowledgment.
  6. Notation and Terminology.
  7. Approximation
  8. Optimization
  9. Fundamental Perspective
  10. Relaxation Principle
  11. Entropy-Regularized Relaxation Principle
  12. Recovering Primal Solutions from Dual Solutions
  13. Algorithms
  14. Coordinate Descent for the Sum of Bivariates
  15. Linear Programming for the Dual Linear Program
  16. ...and 13 more sections

Key Result

Theorem 2.1

Given $G: \mathbb{N}_{\leq m}^n \to \mathbb{R}$, $m \in \mathbb{N}$, a sum of bivariates $F: \mathbb{N}_{\leq m}^n \to \mathbb{R}$, where $F \equiv \sum_{(i,j) \in \mathcal{E}} f_{i,j}$, $\mathcal{E} = \{(i,j) \in \mathcal{V} \times \mathcal{V} \mid i < j \}$, and $\mathcal{V} = \mathbb{N}_{\leq n}$ where

Figures (7)

  • Figure 1: Hierarchy of Optimization Problems: The reducibility results considered in this work. An arrow symbolizes that solving one problem yields the optimal value of another problem. Half arrows are approximations. Dotted arrows are conditional. Known generic problems are set in cursive font.
  • Figure 2: Visualization of a function $L^2$-orthogonal to the sums of bivariates on $\Omega = [0,1]^3$.
  • Figure 3: Visualization of the variable dependencies of the sum of bivariates of \ref{['ex:dynprog']}.
  • Figure 4: Geometric visualization of \ref{['ex:counterrecon']}. Measures $\mu_{1,2}, \mu_{1,3}$, and $\mu_{2,3}$ can be understood to be defined on the respective face of the cube. The relaxed objective function is the sum of the scalar product of each measure with the respective face.
  • Figure 5: Minimization of random sums of bivariates. Benchmark of several algorithms for the optimization of sums of bivariates with $100$ arguments that can each take $5-15$ values for various densities of the Erdős--Rényi random graph that indexes the sum of bivariates. The plots represent the average of $10$ independent runs and of the best value found after given wall-clock time. Primal and dual relaxed values are shown. See \ref{['sec:firstexp']} for details.
  • ...and 2 more figures

Theorems & Definitions (45)

  • Definition 1.1: Sum of Bivariates
  • Theorem 2.1: Approximation
  • proof
  • Corollary 2.1: Dual Variables
  • Remark 1
  • proof
  • Example 3.1: The Hamiltonian Cycle Problem as a Sum of Bivariates savchynskyy2019discrete
  • Example 3.2: Sums of Bivariates as Integer Linear Programs
  • Theorem 3.1: Dynamic Programming for Tree-Indexed Instances
  • proof
  • ...and 35 more