ConstraintLLM: A Neuro-Symbolic Framework for Industrial-Level Constraint Programming

TL;DR

ConstraintLLM presents a neuro-symbolic framework for industrial-level constraint programming by coupling an open-source LLM with a constraint-aware retrieval module, a Tree-of-Thoughts guided exploration, and iterative self-correction, all fine-tuned with multi-instruction supervision. The IndusCP benchmark of 140 NP-hard CP problems enables rigorous evaluation across diverse domains, where ConstraintLLM achieves state-of-the-art solving accuracy and outperforms baselines by large margins, even with smaller LLM backbones. The approach demonstrates strong cross-domain generalization and robustness, owing to constraint-profile based retrieval and structured search, and highlights the practical viability of automated CP modeling for real-world industrial problems. The work provides a scalable path toward automated, trustworthy neuro-symbolic CP modeling with meaningful benchmarks and open-source resources for the community.

Abstract

Constraint programming (CP) is a crucial technology for solving real-world constraint optimization problems (COPs), with the advantages of rich modeling semantics and high solving efficiency. Using large language models (LLMs) to generate formal modeling automatically for COPs is becoming a promising approach, which aims to build trustworthy neuro-symbolic AI with the help of symbolic solvers. However, CP has received less attention compared to works based on operations research (OR) models. We introduce ConstraintLLM, the first LLM specifically designed for CP modeling, which is trained on an open-source LLM with multi-instruction supervised fine-tuning. We propose the Constraint-Aware Retrieval Module (CARM) to increase the in-context learning capabilities, which is integrated in a Tree-of-Thoughts (ToT) framework with guided self-correction mechanism. Moreover, we construct and release IndusCP, the first industrial-level benchmark for CP modeling, which contains 140 challenging tasks from various domains. Our experiments demonstrate that ConstraintLLM achieves state-of-the-art solving accuracy across multiple benchmarks and outperforms the baselines by 2x on the new IndusCP benchmark. Code and data are available at: https://github.com/william4s/ConstraintLLM.

Figures (6)

• Figure 1: The framework of ConstraintLLM. (a) Illustrates the process of constructing IndusCP and preparing the training data. (b) Depicts the training phase, where we employ multi-instruction SFT to teach the model skills in CP modeling, constraint type extraction, and self-correction. (c) Outlines the inference pipeline of ConstraintLLM: a problem, augmented with ICL examples provided by CARM, is fed to the LLM to formulate a CP model. This CP model is then processed by the solver. The feedback from the solver is used to either derive the final answer or to initiate a self-correction process.
• Figure 2: Contribution analysis of key ConstraintLLM components: Comparing the impact of SFT, CARM vs. RAG, and Self-Correction on SA.
• Figure 3: Four representative benchmark examples from our evaluation, showing the diversity of problem types.
• Figure 4: The full one-shot prompt used for the Chain-of-Thought (CoT) baseline.
• Figure 5: The prompt template used for the Retrieval-Augmented Generation (RAG) baseline.