Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Model enabled Mathematical Modeling

Guoyun Zhang

TL;DR

This work probes the feasibility of using a cost-efficient LLM, DeepSeek-R1, to translate natural-language optimization problems into solver-ready mathematical formulations for supply-chain OR. It introduces a structured mitigation suite—LLM-as-a-Judge, Few-shot Learning, Tool Calling, and a Multi-agent Framework—to reduce hallucinations and improve formulation accuracy across four benchmarks (NL4OPT, IndustryOR, EasyLP, ComplexOR). The study demonstrates substantial gains from self-evaluation (LLM-as-a-Judge) especially on NL4OPT and IndustryOR, while revealing trade-offs and limitations in attention to upstream reasoning and tool usage. The findings offer a practical, cost-effective pathway for AI-augmented optimization with clear directions for tool integration, interpretability, and future industrial deployment.

Abstract

The integration of Large Language Models (LLMs) with optimization modeling offers a promising avenue for advancing decision-making in operations research (OR). Traditional optimization methods,such as linear programming, mixed integer programming, and simulation depend heavily on domain expertise to translate real-world problems into solvable mathematical models. While solvers like Gurobi and COPT are powerful, expert input remains essential for defining objectives, constraints, and variables. This research investigates the potential of LLMs, specifically the DeepSeek-R1 model, to bridge this formulation gap using natural language understanding and code generation. Although prior models like GPT-4, Claude, and Bard have shown strong performance in NLP and reasoning tasks, their high token costs and tendency toward hallucinations limit real-world applicability in supply chain contexts. In contrast, DeepSeek-R1, a cost-efficient and high-performing model trained with reinforcement learning, presents a viable alternative. Despite its success in benchmarks such as LiveCodeBench and Math-500, its effectiveness in applied OR scenarios remains under explored. This study systematically evaluates DeepSeek-R1 across four key OR benchmarks: NL4OPT, IndustryOR, EasyLP, and ComplexOR. Our methodology includes baseline assessments, the development of a hallucination taxonomy, and the application of mitigation strategies like LLM-as-a-Judge, Few-shot Learning (FSL), Tool Calling, and a Multi-agent Framework. These techniques aim to reduce hallucinations, enhance formulation accuracy, and better align model outputs with user intent.

Large Language Model enabled Mathematical Modeling

TL;DR

This work probes the feasibility of using a cost-efficient LLM, DeepSeek-R1, to translate natural-language optimization problems into solver-ready mathematical formulations for supply-chain OR. It introduces a structured mitigation suite—LLM-as-a-Judge, Few-shot Learning, Tool Calling, and a Multi-agent Framework—to reduce hallucinations and improve formulation accuracy across four benchmarks (NL4OPT, IndustryOR, EasyLP, ComplexOR). The study demonstrates substantial gains from self-evaluation (LLM-as-a-Judge) especially on NL4OPT and IndustryOR, while revealing trade-offs and limitations in attention to upstream reasoning and tool usage. The findings offer a practical, cost-effective pathway for AI-augmented optimization with clear directions for tool integration, interpretability, and future industrial deployment.

Abstract

The integration of Large Language Models (LLMs) with optimization modeling offers a promising avenue for advancing decision-making in operations research (OR). Traditional optimization methods,such as linear programming, mixed integer programming, and simulation depend heavily on domain expertise to translate real-world problems into solvable mathematical models. While solvers like Gurobi and COPT are powerful, expert input remains essential for defining objectives, constraints, and variables. This research investigates the potential of LLMs, specifically the DeepSeek-R1 model, to bridge this formulation gap using natural language understanding and code generation. Although prior models like GPT-4, Claude, and Bard have shown strong performance in NLP and reasoning tasks, their high token costs and tendency toward hallucinations limit real-world applicability in supply chain contexts. In contrast, DeepSeek-R1, a cost-efficient and high-performing model trained with reinforcement learning, presents a viable alternative. Despite its success in benchmarks such as LiveCodeBench and Math-500, its effectiveness in applied OR scenarios remains under explored. This study systematically evaluates DeepSeek-R1 across four key OR benchmarks: NL4OPT, IndustryOR, EasyLP, and ComplexOR. Our methodology includes baseline assessments, the development of a hallucination taxonomy, and the application of mitigation strategies like LLM-as-a-Judge, Few-shot Learning (FSL), Tool Calling, and a Multi-agent Framework. These techniques aim to reduce hallucinations, enhance formulation accuracy, and better align model outputs with user intent.
Paper Structure (28 sections, 10 equations, 5 figures, 4 tables)

This paper contains 28 sections, 10 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Large Language Model (LLM)
  3. Optimization in Supply Chain
  4. IndustryOR
  5. NL4OPT
  6. ComplexOR
  7. Mamo-EasyLP
  8. COPT
  9. LLMs and Optimization
  10. Results from prior studies
  11. Methodology
  12. Experiments and Results
  13. Evaluation Metrics
  14. Preliminary result
  15. Ablation study for Mitigation
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Evolution of LLMs zhao2023survey
  • Figure 2: GPT-2 Architecture yang2023fluid
  • Figure 3: Error type distribution across four datasets
  • Figure 4: Comparison of error distributions across IndustryOR and NL4OPT benchmarks.
  • Figure 5: Comparison of error distributions across EasyLP and ComplexOR benchmarks.