Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CHORUS: Zero-shot Hierarchical Retrieval and Orchestration for Generating Linear Programming Code

Tasnim Ahmed, Salimur Choudhury

TL;DR

This work tackles the challenge of generating solver-specific LP code from natural language by introducing CHORUS, a retrieval-augmented generation framework. CHORUS combines hierarchical tree-based theory indexing with metadata-augmented code retrieval, two-stage context retrieval, cross-encoder reranking, and expert prompting to produce Gurobi Python LP code with a structured reasoning trace. On the NL4Opt-Code benchmark, open-source LLMs with CHORUS substantially outperform baselines and approach GPT-4-level performance while using far less compute, with notable gains in accuracy and syntactic validity. Ablation studies confirm the critical roles of expert prompting, hierarchical chunking, and structured reasoning, and the framework remains solver- and model-agnostic, enabling adaptation to future solvers and models.

Abstract

Linear Programming (LP) problems aim to find the optimal solution to an objective under constraints. These problems typically require domain knowledge, mathematical skills, and programming ability, presenting significant challenges for non-experts. This study explores the efficiency of Large Language Models (LLMs) in generating solver-specific LP code. We propose CHORUS, a retrieval-augmented generation (RAG) framework for synthesizing Gurobi-based LP code from natural language problem statements. CHORUS incorporates a hierarchical tree-like chunking strategy for theoretical contents and generates additional metadata based on code examples from documentation to facilitate self-contained, semantically coherent retrieval. Two-stage retrieval approach of CHORUS followed by cross-encoder reranking further ensures contextual relevance. Finally, expertly crafted prompt and structured parser with reasoning steps improve code generation performance significantly. Experiments on the NL4Opt-Code benchmark show that CHORUS improves the performance of open-source LLMs such as Llama3.1 (8B), Llama3.3 (70B), Phi4 (14B), Deepseek-r1 (32B), and Qwen2.5-coder (32B) by a significant margin compared to baseline and conventional RAG. It also allows these open-source LLMs to outperform or match the performance of much stronger baselines-GPT3.5 and GPT4 while requiring far fewer computational resources. Ablation studies further demonstrate the importance of expert prompting, hierarchical chunking, and structured reasoning.

CHORUS: Zero-shot Hierarchical Retrieval and Orchestration for Generating Linear Programming Code

TL;DR

This work tackles the challenge of generating solver-specific LP code from natural language by introducing CHORUS, a retrieval-augmented generation framework. CHORUS combines hierarchical tree-based theory indexing with metadata-augmented code retrieval, two-stage context retrieval, cross-encoder reranking, and expert prompting to produce Gurobi Python LP code with a structured reasoning trace. On the NL4Opt-Code benchmark, open-source LLMs with CHORUS substantially outperform baselines and approach GPT-4-level performance while using far less compute, with notable gains in accuracy and syntactic validity. Ablation studies confirm the critical roles of expert prompting, hierarchical chunking, and structured reasoning, and the framework remains solver- and model-agnostic, enabling adaptation to future solvers and models.

Abstract

Linear Programming (LP) problems aim to find the optimal solution to an objective under constraints. These problems typically require domain knowledge, mathematical skills, and programming ability, presenting significant challenges for non-experts. This study explores the efficiency of Large Language Models (LLMs) in generating solver-specific LP code. We propose CHORUS, a retrieval-augmented generation (RAG) framework for synthesizing Gurobi-based LP code from natural language problem statements. CHORUS incorporates a hierarchical tree-like chunking strategy for theoretical contents and generates additional metadata based on code examples from documentation to facilitate self-contained, semantically coherent retrieval. Two-stage retrieval approach of CHORUS followed by cross-encoder reranking further ensures contextual relevance. Finally, expertly crafted prompt and structured parser with reasoning steps improve code generation performance significantly. Experiments on the NL4Opt-Code benchmark show that CHORUS improves the performance of open-source LLMs such as Llama3.1 (8B), Llama3.3 (70B), Phi4 (14B), Deepseek-r1 (32B), and Qwen2.5-coder (32B) by a significant margin compared to baseline and conventional RAG. It also allows these open-source LLMs to outperform or match the performance of much stronger baselines-GPT3.5 and GPT4 while requiring far fewer computational resources. Ablation studies further demonstrate the importance of expert prompting, hierarchical chunking, and structured reasoning.
Paper Structure (26 sections, 5 equations, 3 figures, 2 tables)

This paper contains 26 sections, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. CHORUS
  3. Document Structuring and Indexing
  4. Hierarchical Tree Indexing of Theoretical Content
  5. Metadata-Augmented Indexing of Code Examples
  6. Two-Stage Context Retrieval
  7. Hierarchical Context Retrieval
  8. Code Example Retrieval
  9. Cross-Encoder Reranking for Contextual Relevance
  10. LLM Code Generation
  11. Expert Prompting
  12. Structured Output with Improved Reasoning
  13. Experiments
  14. Task Formulation
  15. Dataset and Metrics
  16. ...and 11 more sections

Figures (3)

  • Figure 1: CHORUS framework for generating Gurobi solver code from LP descriptions. It retrieves and reranks relevant conceptual and implementation-level documents, supplements them with an expert prompt, and employs a structured output parser for automated execution and improved reasoning.
  • Figure 2: Token length distributions for (left) theoretical documentation chunks and (right) code examples.
  • Figure 3: Word frequency analysis contrasting (left) raw code token prevalence with (right) curated metadata terminology from summaries and keywords.