Discrete and Continuous Difference of Submodular Minimization

TL;DR

This work studies the minimization of $F(x)=G(x)-H(x)$ where $G,H$ are normalized submodular on product domains, addressing both discrete and continuous settings. It proves that any function on a discrete domain and any smooth function on a continuous domain admit a DS decomposition, enabling a DC reformulation via the Lovász extension and a corresponding DC algorithm. The authors propose a discrete-domain DS solver (DCA-LS) with $O(1/T)$ convergence to a local minimum and show how to apply the method to continuous domains via discretization, accompanied by theoretical guarantees. Empirically, the approach improves over baselines on integer least squares and integer compressed sensing, while remaining practical in runtime. Overall, the work tightly links DS representations with DC programming and provides actionable algorithms for mixed-integer non-convex optimization problems.

Abstract

Submodular functions, defined on continuous or discrete domains, arise in numerous applications. We study the minimization of the difference of two submodular (DS) functions, over both domains, extending prior work restricted to set functions. We show that all functions on discrete domains and all smooth functions on continuous domains are DS. For discrete domains, we observe that DS minimization is equivalent to minimizing the difference of two convex (DC) functions, as in the set function case. We propose a novel variant of the DC Algorithm (DCA) and apply it to the resulting DC Program, obtaining comparable theoretical guarantees as in the set function case. The algorithm can be applied to continuous domains via discretization. Experiments demonstrate that our method outperforms baselines in integer compressive sensing and integer least squares.

Figures (17)

• Figure 1: Performance results, averaged over 100 runs, for the integer least squares with $\textnormal{SNR}_{\textnormal{dB}} = 20$ dB, $n=100$ (top) and for integer compressed sensing with $\textnormal{SNR}_{\textnormal{dB}} = 8$ dB, $n=256$ and $s=26 = \lceil 0.1n \rceil$ (bottom).
• Figure 2: Results averaged over 50 runs comparing the direct minimization of $F$ on the lattice $\mathcal{X}$ (Lattice) and the minimization of its reduction $\tilde{F}$ on $\{0,1\}^{n \times (k-1)}$ (Reduction) on Problem \ref{['eq: reduction-objective']}.
• Figure 3: Performance results, averaged over 100 runs, for integer least squares experiments: Varying $m$ with $n=400$ and $\textnormal{SNR}_{\textnormal{dB}}=20$ (top), varying $\textnormal{SNR}_{\textnormal{dB}}$ with $m=n=100$ (middle), and varying $\textnormal{SNR}_{\textnormal{dB}}$ with $m=n=400$ (bottom).
• Figure 4: Running times (log-scale), averaged over 100 runs, for integer least squares experiments: (a) Varying $m$ with $n=100$ and $\textnormal{SNR}_{\textnormal{dB}}=20$, (b) varying $m$ with $n=400$ and $\textnormal{SNR}_{\textnormal{dB}}=20$, (c) varying $\textnormal{SNR}_{\textnormal{dB}}$ and $m=n=100$, and (d) varying $\textnormal{SNR}_{\textnormal{dB}}$ and $m=n=400$. Optimal solution is computed using Gurobi.
• Figure 5: Performance results, averaged over 100 runs, for integer compressed sensing experiments with $n=256$: Varying $m$ with $s=13 = \lceil 0.05n \rceil$ and $\textnormal{SNR}_{\textnormal{dB}} = 8$ dB (top), varying $\textnormal{SNR}_{\textnormal{dB}}$ with $s=13 = \lceil 0.05n \rceil$ and $m/n = 0.5$ (middle), and varying $\textnormal{SNR}_{\textnormal{dB}}$ with $s=26 = \lceil 0.1n \rceil$ and $m/n = 0.5$ (bottom).

Theorems & Definitions (55)

• Proposition 2.1
• Definition 2.2
• Proposition 2.3
• Proposition 2.4
• Proposition 2.5
• Proposition 3.0
• proof : Proof Sketch
• Example 3.0
• proof : Proof Sketch
• Proposition 3.0