LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation

TL;DR

This work tackles the challenge of generating optimization programs from problem prompts by fine-tuning general LLMs on a domain specific instruction set. It introduces LLaMoCo, a two phase learning approach that combines a contrastive warm-up with instruction tuning on a large code to code dataset mapping optimization problems to executable optimizers. Key contributions include constructing a 32k+ example instruction set with diverse problem descriptions, data augmentation through Python and LaTeX rephrasings, and demonstrating superior optimization performance over GPT-4 Turbo and other baselines on synthetic and realistic tasks. The results suggest that domain specific instruction tuning can yield robust, single round optimization code generation with practical efficiency gains, even with relatively small LLMs.

Abstract

Recent research explores optimization using large language models (LLMs) by either iteratively seeking next-step solutions from LLMs or directly prompting LLMs for an optimizer. However, these approaches exhibit inherent limitations, including low operational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. Specifically, we establish a comprehensive instruction set containing well-described problem prompts and effective optimization codes. We then develop a novel two-phase learning strategy that incorporates a contrastive learning-based warm-up procedure before the instruction-tuning phase to enhance the convergence behavior during model fine-tuning. The experiment results demonstrate that a CodeGen (350M) model fine-tuned by our LLaMoCo achieves superior optimization performance compared to GPT-4 Turbo and the other competitors across both synthetic and realistic problem sets. The fine-tuned model and the usage instructions are available at https://anonymous.4open.science/r/LLaMoCo-722A.

Figures (10)

• Figure 1: Conceptual overview of LLMs as optimizers. Left: optimization through iteratively prompting LLMs for better solutions (solution-to-solution style), such as OPRO opro. Middle: optimization through directly prompting LLMs for an optimizer with code implementation, such as OptiMus optimus. To ensure rational output, the prompts should include hints about the type of problem and the suggested optimizer. Right: our LLaMoCo, which first tunes general LLMs on a problem-code instruction set, then can be used to generate proper optimization code given the formatted problem prompts.
• Figure 2: Input-output example of our instruction set. For a given problem (red box), diverse task descriptions in Python/LaTeX formats construct the prompt (yellow box). The code implementation of an effective optimizer is provided as the answer (green box).
• Figure 3: The workflow of contrastive warm-up strategy. Given an anchor problem prompt, we collect its similar prompts and dissimilar prompts within a mini-batch. The positive and negative sample losses are calculated on the latent space representations.
• Figure 4: Effectiveness of the contrastive warm-up strategy on training curves (Left) and performance gains (Right).
• Figure 5: Effectiveness of the diversity enhancement strategy (Left) and the data distribution balancing strategy (Right).