OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models

TL;DR

OptiMUS tackles the automation gap in optimization modeling by deploying a modular, multi-agent LLM system that translates natural-language descriptions into structured MILP/LP formulations, generates and debugs solver code, and validates results with data-driven evaluation. A central connection graph keeps context and dependencies local, enabling scalable processing of long texts and large data files. The approach yields state-of-the-art performance on existing benchmarks and on the new NLP4LP dataset, with substantial gains compared to baselines and detailed analyses of ablations, data handling, and failure modes. By coupling LLM-driven modeling with traditional solvers (e.g., $\text{MILP}$/LP) and releasing NLP4LP, OptiMUS points toward scalable, accessible optimization tooling that can benefit industry sectors with limited access to optimization expertise.

Abstract

Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the-art solvers because the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. This paper introduces OptiMUS, a Large Language Model (LLM)-based agent designed to formulate and solve (mixed integer) linear programming problems from their natural language descriptions. OptiMUS can develop mathematical models, write and debug solver code, evaluate the generated solutions, and improve its model and code based on these evaluations. OptiMUS utilizes a modular structure to process problems, allowing it to handle problems with long descriptions and complex data without long prompts. Experiments demonstrate that OptiMUS outperforms existing state-of-the-art methods on easy datasets by more than $20\%$ and on hard datasets (including a new dataset, NLP4LP, released with this paper that features long and complex problems) by more than $30\%$.

Figures (7)

• Figure 1: OptiMUS uses a structured sequence of LLM agents to effectively model and solve optimization problems. First, the natural language representation of the problem is preprocessed into a structured problem. Then, a team of agents iteratively augments the structured problem with a connection graph, mathematical formulations of each clause, and code for each clause. The agents continue work until the problem is solved. (Dashed lines represent communications that can occur multiple times.)
• Figure 2: OptiMUS preprocesses natural language representations of a problem into a modular state. The components of the modular state are: 1) parameters and their shape, 2) objective, 3) background and context, and 4) implicit and explicit constraints.
• Figure 3: The formulation process for a single constraint. The formulation agent identifies any parameters and variables appearing in the constraint, including new variables that it may need to define. It defines new variables as needed, updates the connection graph which records which constraints use which parameters and which variables, and annotates the constraint with a LaTeX formulation. (dashed lines represent new connections and variables)
• Figure 4: OptiMUS uses the connection graph to extract and use only the relevant context in each prompt. In this example, the programmer agent fetches the context via the connection graph to debug a bogus constraint code. Without the graph, the LLM would have needed to process the whole code, including the code for the other parameters, variables, constraints, and the objective.
• Figure 5: OptiMUS can solve more problems on difficult datasets (ComplexOR, NLP4OPT) when more agent calls are allowed, demonstrating the importance of self-improvement.