DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking

TL;DR

This work targets the automation of engineering design under multiple real-world constraints, a setting where prior RAG systems struggle. It introduces SolutionBench, a domain-diverse benchmark built from authoritative reports, and SolutionRAG, a system that combines tree-based exploration with bi-point thinking to iteratively improve design and review, while pruning to balance efficiency. Empirical results show SolutionRAG achieving state-of-the-art performance across eight engineering domains, outperforming deep reasoning and traditional RAG baselines and illustrating the value of structured, critique-informed reasoning. The work highlights the potential for more reliable, scalable automated engineering design, while acknowledging limitations in data scope and computational resources and suggesting avenues for reinforcement learning and hyperparameter exploration.

Abstract

Designing solutions for complex engineering challenges is crucial in human production activities. However, previous research in the retrieval-augmented generation (RAG) field has not sufficiently addressed tasks related to the design of complex engineering solutions. To fill this gap, we introduce a new benchmark, SolutionBench, to evaluate a system's ability to generate complete and feasible solutions for engineering problems with multiple complex constraints. To further advance the design of complex engineering solutions, we propose a novel system, SolutionRAG, that leverages the tree-based exploration and bi-point thinking mechanism to generate reliable solutions. Extensive experimental results demonstrate that SolutionRAG achieves state-of-the-art (SOTA) performance on the SolutionBench, highlighting its potential to enhance the automation and reliability of complex engineering solution design in real-world applications.

Figures (8)

• Figure 1: This paper proposes the complex engineering solution design task and a new system that can generate reliable solutions via the bi-point thinking tree.
• Figure 2: Illustration of the SolutionBench construction method, which includes collecting technology reports from engineering journals to ensure authority and authenticity, extracting useful content based on a manually formatted template and powerful LLMs, and finally harvesting the benchmark after manual verification and merging.
• Figure 3: Illustration of SolutionRAG, we set the child number of each node as 2 for easy presentation above. SolutionRAG uses tree-based exploration to find optimal solution improvement process, bi-point thinking to guarantee generated solutions satisfy all constraints, and a pruning mechanism to balance efficiency and performance.
• Figure 4: Performance changes during the tree growth. The figure shows that scores become higher as the tree grows, proving SolutionRAG can indeed improve the solution scores as inference being deep.
• Figure 5: Effectiveness of node evaluation mechanism. The figure shows that scores in retained nodes are higher than in pruned nodes, thus the node evaluation is an effective method for judging and pruning in SolutionRAG.