HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows

TL;DR

A novel framework HDFlow for complex reasoning with LLMs that combines fast and slow thinking modes in an adaptive manner and hybrid thinking achieves the highest accuracy while providing an effective balance between computational efficiency and performance is proposed.

Abstract

Despite recent advancements in large language models (LLMs), their performance on complex reasoning problems requiring multi-step thinking and combining various skills is still limited. To address this, we propose a novel framework HDFlow for complex reasoning with LLMs that combines fast and slow thinking modes in an adaptive manner. Our approach consists of two key components: 1) a new approach for slow, deliberate reasoning called Dynamic Workflow, which automatically decomposes complex problems into more manageable sub-tasks and dynamically designs a workflow to assemble specialized LLM or symbolic reasoning tools to solve sub-tasks; 2) Hybrid Thinking, a general framework that dynamically combines fast and slow thinking based on problem complexity. Finally, we propose an easy-to-scale method for automatically synthesizing a large-scale dataset of 27K challenging reasoning problems for complex reasoning and a hybrid thinking tuning method that trains smaller LLMs on this dataset to internalize the fast/slow hybrid reasoning strategies. Experiments on four reasoning benchmark datasets demonstrate that our slow thinking with dynamic workflows significantly outperforms Chain-of-Thought, and hybrid thinking achieves the highest accuracy while providing an effective balance between computational efficiency and performance. Fine-tuning using our hybrid thinking approach also significantly boosts the complex reasoning capabilities of open-source language models. The results showcase the promise of slow thinking, dynamic workflows, and hybrid thinking in expanding the frontier of complex problem-solving with LLMs\footnote{Code and data will be released at \url{https://github.com/wenlinyao/HDFlow}.}.

Figures (5)

• Figure 1: Overview of our HDFlow approach for complex problem-solving. Overall, it is a dual-path hybrid thinking approach, beginning with a CoT solver for initial fast reasoning followed by verification of each reasoning step. If verification fails, the process transitions to a slower, more deliberate "Dynamic Workflow Solver." This solver iterates until a verified answer is obtained, incorporating a final verification step before concluding with a solution.
• Figure 2: Three-Stage Framework of Dynamic Workflow. The dynamic workflow design begins with Problem Reflection, where key elements are analyzed and sub-tasks identified. Stage 2 focuses on Expert Design, utilizing a variety of specialists and tools to architect an optimal workflow. Stage 3 involves constructing and executing the workflow graph to get the final result.
• Figure 3: Data Synthesis of Complex Reasoning Problems. The creation and refinement of reasoning problems contain three steps. Step 1 involves brainstorming and generating high-level descriptions of new reasoning tasks, either inspired by human-written tasks or directly writing puzzle tasks. Step 1 produces 45K descriptions of reasoning tasks. Step 2 performs semantic matching and deduplication and results in 18K reasoning task descriptions. The final Step 3 writes concrete questions based on task descriptions and applies a CoT validation process to filter or refine the tasks down to 27k valid reasoning problems.
• Figure 4: Three example reasoning problems generated by our data synthesis approach.
• Figure 5: Proportion of fast thinking (CoT) and slow thinking (dynamic workflow) applied in hybrid thinking across four datasets. The left is GPT-4-Turbo (performance is shown in Table \ref{['tab:gpt4_perf']}), while the right is Llama-3-8B-Instruct after our hybrid thinking tuning (Table \ref{['tab:llama3_perf']}).