BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solving

TL;DR

This work tackles the gap in open-source operations research datasets by introducing StructuredOR, a dataset that provides not only objective values but full annotations of the modeling process to enable reinforcement learning-based reasoning. It then presents BPP-Search, a novel algorithm that combines Beam Search, a Process Reward Model (PRM), and a Pairwise Preference model to efficiently navigate Tree-of-Thought reasoning for mathematical modeling tasks. Across StructuredOR, NL4OPT, and MAMO-ComplexLP, BPP-Search demonstrates higher accuracy with fewer reasoning steps than state-of-the-art baselines, addressing both correctness and computational efficiency. The approach advances RL-enabled mathematical modeling in OR and offers a scalable framework for robust final-candidate selection in ToT, with practical implications for automated problem solving in logistics, scheduling, and networks. The work also discusses trade-offs in tree structure and the gap between LLM capability and OR problem scale, guided by a standardized data format and extensive ablations.

Abstract

LLMs exhibit advanced reasoning capabilities, offering the potential to transform natural language questions into mathematical models. However, existing open-source datasets in operations research domain lack detailed annotations of the modeling process, such as variable definitions, focusing solely on objective values, which hinders reinforcement learning applications. To address this, we release the StructuredOR dataset, annotated with comprehensive labels that capture the complete mathematical modeling process. We further propose BPP-Search, an algorithm that integrates reinforcement learning into a tree-of-thought structure using Beam search, a Process reward model, and a pairwise Preference algorithm. This approach enables efficient exploration of tree structures, avoiding exhaustive search while improving accuracy. Extensive experiments on StructuredOR, NL4OPT, and MAMO-ComplexLP datasets show that BPP-Search significantly outperforms state-of-the-art methods. In tree-based reasoning, BPP-Search excels in accuracy and efficiency, enabling faster retrieval of correct solutions. The StructuredOR dataset is available on Huggingface https://huggingface.co/datasets/LLM4OR/StructuredOR and GitHub https://github.com/LLM4OR/StructuredOR.

Figures (8)

• Figure 1: Reasoning steps. The process follows the path $Q \to S \to P \to V \to O \to C$, where $Q$, $S$, $P$, $V$, $O$ and $C$ represent the question, set, parameter, variable, objective, and constraint respectively.
• Figure 2: The structure of the Tree of Thought. Here, Q represents the question, SP represents set and parameter, V represents variable, and OC represents objective and constraint.
• Figure 3: Pipeline of the construction process of our proposed StructuredOR dataset.
• Figure 4: A real demonstration of the BPP-Search process with a beam search width of 2. Yellow nodes represent pruned nodes that are not explored, while light blue nodes indicate nodes that have been visited.
• Figure 5: An example showcasing a concrete question and its structured modeling process as the label in the StructuredOR dataset.