Hard Constraints Meet Soft Generation: Guaranteed Feasibility for LLM-based Combinatorial Optimization
Yang Liu, Chuan Zhou, Yancheng Chen, Shuai Zhang, Xixun Lin, Xiaoqing Wang
TL;DR
This work tackles the lack of feasibility guarantees in LLM-based combinatorial optimization by introducing FALCON, a framework that guarantees 100% feasibility through a three-layer architecture: grammar-constrained decoding for syntactic validity, a semantic feasibility repair layer, and adaptive Best-of-$N$ sampling to allocate inference effort. It advances training with Best-anchored Objective-guided Preference Optimization (BOPO), proving convergence $O(1/\sqrt{T})$ and bounding repair-induced quality loss. The approach is evaluated across seven NP-hard CO problems, where FALCON achieves perfect feasibility while maintaining competitive or superior solution quality compared with state-of-the-art neural and LLM-based solvers, and with substantial efficiency gains from adaptive sampling. The combination of formal feasibility guarantees and practical performance suggests strong potential for deploying LLM-based CO solvers in safety-critical or high-stakes applications, providing reliable and scalable decision support across domains.
Abstract
Large language models (LLMs) have emerged as promising general-purpose solvers for combinatorial optimization (CO), yet they fundamentally lack mechanisms to guarantee solution feasibility which is critical for real-world deployment. In this work, we introduce FALCON, a framework that ensures 100\% feasibility through three key innovations: (i) \emph{grammar-constrained decoding} enforces syntactic validity, (ii) a \emph{feasibility repair layer} corrects semantic constraint violations, and (iii) \emph{adaptive Best-of-$N$ sampling} allocates inference compute efficiently. To train the underlying LLM, we introduce the Best-anchored Objective-guided Preference Optimization (BOPO) in LLM training, which weights preference pairs by their objective gap, providing dense supervision without human labels. Theoretically, we prove convergence for BOPO and provide bounds on repair-induced quality loss. Empirically, across seven NP-hard CO problems, FALCON achieves perfect feasibility while matching or exceeding the solution quality of state-of-the-art neural and LLM-based solvers.