Effective Generation of Feasible Solutions for Integer Programming via Guided Diffusion

TL;DR

This work tackles the challenge of rapidly producing complete feasible solutions for Integer Programming by learning end-to-end from IP instances to solutions. It introduces Contrastive IP-Solution Pre-training to align instance and solution embeddings, then models the distribution of solution embeddings with conditioned diffusion (DDPM/DDIM), coupled with a decoder to recover discrete solutions. The key innovation is IP-guided diffusion sampling, which integrates constraint and objective information into the sampling process, yielding high-feasibility complete solutions (>$89.7\%$) with quality comparable to solver heuristics, and even surpassing state-of-the-art when combined with SCIP's CompleteSol. The framework demonstrates strong generalization to larger instances, ablation-confirmed effectiveness of guidance components, and a clear potential to move toward solver-free neural IP solving in practice, despite diffusion sampling speed being an area for acceleration.

Abstract

Feasible solutions are crucial for Integer Programming (IP) since they can substantially speed up the solving process. In many applications, similar IP instances often exhibit similar structures and shared solution distributions, which can be potentially modeled by deep learning methods. Unfortunately, existing deep-learning-based algorithms, such as Neural Diving and Predict-and-search framework, are limited to generating only partial feasible solutions, and they must rely on solvers like SCIP and Gurobi to complete the solutions for a given IP problem. In this paper, we propose a novel framework that generates complete feasible solutions end-to-end. Our framework leverages contrastive learning to characterize the relationship between IP instances and solutions, and learns latent embeddings for both IP instances and their solutions. Further, the framework employs diffusion models to learn the distribution of solution embeddings conditioned on IP representations, with a dedicated guided sampling strategy that accounts for both constraints and objectives. We empirically evaluate our framework on four typical datasets of IP problems, and show that it effectively generates complete feasible solutions with a high probability (> 89.7 \%) without the reliance of Solvers and the quality of solutions is comparable to the best heuristic solutions from Gurobi. Furthermore, by integrating our method's sampled partial solutions with the CompleteSol heuristic from SCIP, the resulting feasible solutions outperform those from state-of-the-art methods across all datasets, exhibiting a 3.7 to 33.7\% improvement in the gap to optimal values, and maintaining a feasible ratio of over 99.7\% for all datasets.

Figures (5)

• Figure 1: Our method first trains the IP Encoder and Solution Encoder to acquire the IP embedding ($\mathbf{z}_{i}$) and Solution embedding ($\mathbf{z}_{\mathbf{x}}$) using CISP. We then jointly train diffusion models and the solution decoder to capture the distribution of solutions given a specific IP instance. In the sampling stage, we employ an IP guided diffusion sampling to account for both the objective and constraints.
• Figure 2: Diffusion model $\mathbf{f}_{\theta}(\mathbf{z}_{\mathbf{x}}^{(t)}, \mathbf{z}_{i}, t)$
• Figure 3: The sampling results from different methods. For Neural Diving, we present the predicted solution from GCN, random sampling according to the predicted solution, the partial solution obtained via SelectiveNet (only node 1, 9 and 13 are assigned to 0), and the completing result by calling CompleteSol heuristic. For DDIM and IP Guided DDIM, we present the results from different time steps (transformed to solution space by a decoder) during sampling.
• Figure 4: The feasible ratio in 100 instances, with each instance sampled 30 complete or partial solutions. For diffusion model, we measure the feasible ratio of complete solutions. Two versions of Neural Diving are trained with distinct coverage thresholds, referred to as ND (low coverage) and ND (high coverage). The feasibility ratio is evaluated only for partial solutions from Neural Diving, as the complete solutions from this method yield a 0% feasibility ratio.
• Figure 5: The objective distribution of 1000 solutions sampled from a single instance.