On Densest $k$-Subgraph Mining and Diagonal Loading: Optimization Landscape and Finite-Step Exact Convergence Analysis

TL;DR

This work analyzes a diagonal-loading non-convex relaxation for the Densest $k$-Subgraph problem ($D_kS$). It proves tightness for all subgraph sizes when the penalty satisfies $\lambda \ge 1$ and uncovers a benign optimization landscape: integral stationary points are local maxima, while non-integral stationary points are strict saddles for $\lambda > 1$. Building on this geometry, the authors propose a saddle-escaping Frank–Wolfe algorithm (SE-FW) that converges in a finite number of steps to an integral local maximizer by integrating deterministic ascent steps near saddles. They provide a global convergence rate of $O(1/\sqrt{t})$ and bound the number of escapes by $O(k^2 n^2)$, with numerical experiments on real and synthetic graphs confirming finite-step convergence and saddle escaping. The results offer a rigorous theoretical understanding of Lu et al.’s framework and explain its practical effectiveness for large-scale D$k$S problems.

Abstract

The Densest $k$-Subgraph (D$k$S) is a fundamental combinatorial problem known for its theoretical hardness and breadth of applications. Recently, Lu et al. (AAAI 2025) introduced a penalty-based non-convex relaxation that achieves promising empirical performance; however, a rigorous theoretical understanding of its success remains unclear. In this work, we bridge this gap by providing a comprehensive theoretical analysis. We first establish the tightness of the relaxation, ensuring that the global maximum values of the original combinatorial problem and the relaxed problem coincide. Then we reveal the benign geometry of the optimization landscape by proving a strict dichotomy of stationary points: all integral stationary points are local maximizers, whereas all non-integral stationary points are strict saddles with explicit positive curvature. We propose a saddle-escaping Frank--Wolfe algorithm and prove that it achieves exact convergence to an integral local maximizer in a finite number of steps.

Figures (3)

• Figure 1: Visual representation of the main results in Section \ref{['sec:landscape']}. Yellow rectangle: set of all feasible points $\mathcal{C}_k^{n}$ of \ref{['p3']}. Orange set $\mathcal{A}$: subset of all integral points of \ref{['p3']}. Purple set $\mathcal{B}$: subset of all stationary points of \ref{['p3']}. Theorem \ref{['thm3.6']}:$\mathcal{A} \cap \mathcal{B}$ is the set of all local maximizers of \ref{['p3']} when $\lambda > 1$ is non-integral. Theorem \ref{['thm3.8']}:$\mathcal{B} \setminus \mathcal{A}$ is the set of all strict saddle points of \ref{['p3']} when $\lambda > 1$ is non-integral.
• Figure 2: Finite-Step Exact Convergence on the Facebook Dataset. Comparison between our step-size rule (blue) and the step-size rule proposed by lacoste2016convergence (orange). (a): The Frank--Wolfe gap is defined as $G(\boldsymbol{x}) := \max_{\boldsymbol{s} \in \mathcal{C}_{k}^{n}} \langle \nabla g(\boldsymbol{x}), \boldsymbol{s} - \boldsymbol{x} \rangle$, where $\mathcal{C}_{k}^{n} = \{ \boldsymbol{y} \in [0, 1]^{n} \mid \sum_{i \in [n]} y_{i} = k \}$. This metric characterizes the proximity to stationarity, vanishing if and only if $\boldsymbol{x}$ is a stationary point. (b): The integrality is defined as $\Vert \boldsymbol{x} \Vert_{2}^{2} / k$. This metric characterizes the proximity to integrality, attaining one if and only if $\boldsymbol{x}$ is an integral point. (c): The objective value of $g (\boldsymbol{x})$.
• Figure 3: Saddle Escaping on a Synthetic Regular Graph.

Theorems & Definitions (55)

• Theorem 2.1: Corollary of Theorem 2 in lu2025densest
• Theorem 2.2: Lemma 5.1 in barman2018approximating
• Theorem 2.3: Extension of Lemma 5.1 in barman2018approximating
• proof
• Remark 2.4
• Lemma 3.1
• proof
• Theorem 3.2: Monotonicity of Local Maximizers
• proof
• Definition 3.3: Stationary Point