Adaptive Stabilization Based on Machine Learning for Column Generation

TL;DR

This paper addresses slow convergence in column generation due to dual oscillations by introducing ASCG-ML, which combines machine-learned predictions of the optimal dual solution with an adaptive stabilization penalty. The method defines a generalized dual that penalizes deviations from ML predictions and dynamically tunes the penalty via the Lagrangian gap to balance guidance and convergence. Empirical results on graph coloring show substantial improvements in convergence speed and solution times, with ML models (FFNN and GCN) offering complementary strengths across instance families. The approach demonstrates robust generalization and suggests a path to integrating ML more deeply into CG across problems beyond graph coloring, aided by exact and heuristic pricing variants. The work provides both theoretical convergence guarantees and practical evidence of accelerated CG performance.

Abstract

Column generation (CG) is a well-established method for solving large-scale linear programs. It involves iteratively optimizing a subproblem containing a subset of columns and using its dual solution to generate new columns with negative reduced costs. This process continues until the dual values converge to the optimal dual solution to the original problem. A natural phenomenon in CG is the heavy oscillation of the dual values during iterations, which can lead to a substantial slowdown in the convergence rate. Stabilization techniques are devised to accelerate the convergence of dual values by using information beyond the state of the current subproblem. However, there remains a significant gap in obtaining more accurate dual values at an earlier stage. To further narrow this gap, this paper introduces a novel approach consisting of 1) a machine learning approach for accurate prediction of optimal dual solutions and 2) an adaptive stabilization technique that effectively capitalizes on accurate predictions. On the graph coloring problem, we show that our method achieves a significantly improved convergence rate compared to traditional methods.

Figures (5)

• Figure 1: The trajectory of dual iterates for our adaptive stabilized CG based on ML (ASCG-ML), compared to CG and stabilized CG (SCG) for the first $50$ iterations. Each point (or a dual iterate) represents a set of dual values at an iteration of CG. The axes are the first two principal components of the high-dimensional dual space. The test graphs DSJC125.5 and myciel5 are from standard benchmarks for the graph coloring problem.
• Figure 2: The trend of penalty value $\epsilon$ computed by exact or heuristic pricing in different iterations of ASCG, both bounded by the Lagrangian Gap on several Graph Coloring Benchmarks.
• Figure 3: The number of solved runs within an iteration limit on the Matilda graphs.
• Figure 4: Density histogram of the $1326$ Matilda graphs.
• Figure 5: Performance of the compared methods with different penalty values $\epsilon$. The $x$-axis shows the iteration number normalized with respect to the iteration number of CG.

Theorems & Definitions (15)

• Lemma 3.1
• Theorem 3.2
• Lemma 3.3
• Corollary 3.4
• Lemma 3.5
• Lemma 3.1
• proof
• Theorem 3.2
• proof
• Lemma 3.3