DAOpt: Modeling and Evaluation of Data-Driven Optimization under Uncertainty with LLMs

TL;DR

The paper addresses optimization under uncertainty by leveraging large language models (LLMs) within a data-driven, multi-agent framework. It introduces DAOpt, a three-module architecture that (i) builds OptU, a dataset and data-generation environment decoupling problem descriptions from data, (ii) uses a Data-driven Decision-making Module with an RSOME-backed OR Domain Knowledge Learner and a Reflexion-based Checker, and (iii) evaluates solutions via a Data-driven Simulation and Evaluation Module focused on out-of-sample feasibility and robustness. The problem is formulated as $\min_{x\in\mathcal{X}}\rho_0\big[f_0(x,\tilde{\boldsymbol{p}})\big]$ subject to $\rho_n\big[f_n(x,\tilde{\boldsymbol{p}})\big]\le 0$, enabling stochastic, robust, and distributionally robust models. Empirical results on OptU with DeepSeek-V3 and GPT-4o show that DAOpt improves out-of-sample feasibility (e.g., DRO/RO with DAOpt achieving $\text{FR}>0.7$) and reduces the optimizer's curse, validating the benefits of integrating domain knowledge and duality reformulations into LLM-driven optimization under uncertainty.

Abstract

Recent advances in large language models (LLMs) have accelerated research on automated optimization modeling. While real-world decision-making is inherently uncertain, most existing work has focused on deterministic optimization with known parameters, leaving the application of LLMs in uncertain settings largely unexplored. To that end, we propose the DAOpt framework including a new dataset OptU, a multi-agent decision-making module, and a simulation environment for evaluating LLMs with a focus on out-of-sample feasibility and robustness. Additionally, we enhance LLMs' modeling capabilities by incorporating few-shot learning with domain knowledge from stochastic and robust optimization.

Figures (2)

• Figure 1: DAOpt framework
• Figure 2: Deterministic optimization can fail in an uncertain environment