Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Scalable Global Optimization Algorithm For Constrained Clustering

Pedro Chumpitaz-Flores, My Duong, Cristobal Heredia, Kaixun Hua

TL;DR

This work tackles constrained MSSC with pairwise must-link and cannot-link constraints, an NP-hard problem that challenges existing exact methods to small-scale settings. The authors propose SDC-GBB, a deterministic global optimization framework that (i) collapses must-link groups into centroid pseudo-samples, (ii) applies geometric sample-determination to prune cannot-links, and (iii) branches only on centroid variables, ensuring convergence to an ε-optimal solution. They combine a grouped-sample Lagrangian decomposition for tight lower bounds with efficient upper bounds from CL coloring and ML closed-form bounds, enabling parallel, scalable optimization. Empirical results show SDC-GBB solving up to 1,500,000 ML-constrained samples and 200,000 CL-constrained samples with gaps under 3%, outperforming or matching state-of-the-art exact methods on small/medium data and significantly outperforming them on large-scale problems. This delivers deterministic, scalable constrained clustering suitable for diverse real-world applications while highlighting avenues for stronger CL relaxations and broader ethical considerations.

Abstract

Constrained clustering leverages limited domain knowledge to improve clustering performance and interpretability, but incorporating pairwise must-link and cannot-link constraints is an NP-hard challenge, making global optimization intractable. Existing mixed-integer optimization methods are confined to small-scale datasets, limiting their utility. We propose Sample-Driven Constrained Group-Based Branch-and-Bound (SDC-GBB), a decomposable branch-and-bound (BB) framework that collapses must-linked samples into centroid-based pseudo-samples and prunes cannot-link through geometric rules, while preserving convergence and guaranteeing global optimality. By integrating grouped-sample Lagrangian decomposition and geometric elimination rules for efficient lower and upper bounds, the algorithm attains highly scalable pairwise k-Means constrained clustering via parallelism. Experimental results show that our approach handles datasets with 200,000 samples with cannot-link constraints and 1,500,000 samples with must-link constraints, which is 200 - 1500 times larger than the current state-of-the-art under comparable constraint settings, while reaching an optimality gap of less than 3%. In providing deterministic global guarantees, our method also avoids the search failures that off-the-shelf heuristics often encounter on large datasets.

A Scalable Global Optimization Algorithm For Constrained Clustering

TL;DR

This work tackles constrained MSSC with pairwise must-link and cannot-link constraints, an NP-hard problem that challenges existing exact methods to small-scale settings. The authors propose SDC-GBB, a deterministic global optimization framework that (i) collapses must-link groups into centroid pseudo-samples, (ii) applies geometric sample-determination to prune cannot-links, and (iii) branches only on centroid variables, ensuring convergence to an ε-optimal solution. They combine a grouped-sample Lagrangian decomposition for tight lower bounds with efficient upper bounds from CL coloring and ML closed-form bounds, enabling parallel, scalable optimization. Empirical results show SDC-GBB solving up to 1,500,000 ML-constrained samples and 200,000 CL-constrained samples with gaps under 3%, outperforming or matching state-of-the-art exact methods on small/medium data and significantly outperforming them on large-scale problems. This delivers deterministic, scalable constrained clustering suitable for diverse real-world applications while highlighting avenues for stronger CL relaxations and broader ethical considerations.

Abstract

Constrained clustering leverages limited domain knowledge to improve clustering performance and interpretability, but incorporating pairwise must-link and cannot-link constraints is an NP-hard challenge, making global optimization intractable. Existing mixed-integer optimization methods are confined to small-scale datasets, limiting their utility. We propose Sample-Driven Constrained Group-Based Branch-and-Bound (SDC-GBB), a decomposable branch-and-bound (BB) framework that collapses must-linked samples into centroid-based pseudo-samples and prunes cannot-link through geometric rules, while preserving convergence and guaranteeing global optimality. By integrating grouped-sample Lagrangian decomposition and geometric elimination rules for efficient lower and upper bounds, the algorithm attains highly scalable pairwise k-Means constrained clustering via parallelism. Experimental results show that our approach handles datasets with 200,000 samples with cannot-link constraints and 1,500,000 samples with must-link constraints, which is 200 - 1500 times larger than the current state-of-the-art under comparable constraint settings, while reaching an optimality gap of less than 3%. In providing deterministic global guarantees, our method also avoids the search failures that off-the-shelf heuristics often encounter on large datasets.
Paper Structure (28 sections, 8 theorems, 19 equations, 4 figures, 9 tables, 1 algorithm)

This paper contains 28 sections, 8 theorems, 19 equations, 4 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Mixed‑Integer Programming for Pairwise‑Constrained $k$‑Means
  3. Reduced-space Branch-and-Bound Algorithm
  4. Geometric Sample Determination Rules
  5. Equivalent Unconstrained Clustering Problem
  6. Mixed ML and CL constraints.
  7. Upper Bounding Strategies
  8. $K$–Coloring Bound for CL Constraints.
  9. Closed–Form Bound for ML Constraints.
  10. Lower Bounding Strategy with Grouping-based Lagrangian Decomposition
  11. Branch-and-Bound Clustering Scheme
  12. Computational Experiments
  13. Numerical Results
  14. Small and medium-sized datasets
  15. Large and huge datasets
  16. ...and 13 more sections

Key Result

Lemma 3.1

For any sample $s\!\in\!\mathcal{S}$ and any cluster region $M_k$, $d_{\min}(\mathbf{x}_s,M_k)>\rho\;\Longrightarrow\;b_{s,k}=0$ in every optimal solution with objective value not larger than the incumbent.

Figures (4)

  • Figure 1: Illustration of sample‐determination via link propagation for $K=3$.
  • Figure 2: Left: three samples $x_1,x_2,x_3$ with must-link $(x_1,x_2)$ and centroid $\mu$. Right: collapse of $\{x_1,x_2\}$ into two pseudo-samples at $\mu_{ml}=\tfrac{1}{2}(x_1+x_2)$ preserves the optimal centroid $\mu$.
  • Figure 3: Effect of Must–Link Constraints on (a) the number of nodes processed and (b) time per node, for both synthetic datasets.
  • Figure 4: Effect of Cannot–Link Constraints on (a) the number of nodes processed and (b) time per node, for both synthetic datasets.

Theorems & Definitions (13)

  • Lemma 3.1: Early–elimination
  • Lemma 3.2: Forced assignment
  • Lemma 3.3
  • Theorem 3.4
  • Theorem 3.5
  • proof
  • proof
  • Lemma C.1: Lower Bounding Consistency
  • proof
  • Lemma C.2: Lower Bounding Convergence
  • ...and 3 more