Learn to Relax with Large Language Models: Solving Constraint Optimization Problems via Bidirectional Coevolution

Abstract

Large Language Model (LLM)-based optimization has recently shown promise for autonomous problem solving, yet most approaches still cast LLMs as passive constraint checkers rather than proactive strategy designers, limiting their effectiveness on complex Constraint Optimization Problems (COPs). To address this, we present AutoCO, an end-to-end Automated Constraint Optimization method that tightly couples operations-research principles of constraint relaxation with LLM reasoning. A core innovation is a unified triple-representation that binds relaxation strategies, algorithmic principles, and executable codes. This design enables the LLM to synthesize, justify, and instantiate relaxation strategies that are both principled and executable. To navigate fragmented solution spaces, AutoCO employs a bidirectional global-local coevolution mechanism, synergistically coupling Monte Carlo Tree Search (MCTS) for global relaxation-trajectory exploration with Evolutionary Algorithms (EAs) for local solution intensification. This continuous exchange of priors and feedback explicitly balances diversification and intensification, thus preventing premature convergence. Extensive experiments on three challenging COP benchmarks validate AutoCO's consistent effectiveness and superior performance, especially in hard regimes where current methods degrade. Results highlight AutoCO as a principled and effective path toward proactive, verifiable LLM-driven optimization.

Figures (16)

• Figure 1: Comparisons of human/LLM-based solutions for COPs. A) Expert-designed method leverages human analysis to relax constraints for feasible solutions. B) Current LLM-based methods focus on code generation, lacking systematic problem analysis. C) Our AutoCO combines human-inspired relaxation strategies with automation to effectively discover feasible solutions.
• Figure 2: AutoCO (blue) vs. Current LLM method (green) performance on VRPTW-fuel problems over 100 iterations. The autonomous constraint relaxation temporarily expands feasible regions, enhancing optimization feedback.
• Figure 3: Architecture of AutoCO. Initially, we use LLMs to parse user-input problems and generate initial constraint relaxation strategies. Next, the bidirectional coevolution mechanism combining local EA and global MCTS explores and optimizes strategies and codes. Finally, we evaluate the generated algorithms on problem instances and provide individual fitness feedback.
• Figure 4: Time distribution of AutoCO components.
• Figure 5: Effectiveness of various constraint relaxation strategies under different computational budgets on SFL8 and SFL-5 (Dual).