SMaRT: Select, Mix, and ReinvenT -- A Strategy Fusion Framework for LLM-Driven Reasoning and Planning

TL;DR

SMaRT addresses the fragility of single-strategy prompting in LLM-driven reasoning and planning by introducing a two-stage strategy fusion framework that lets LLMs act as intelligent integrators across multiple base strategies. Stage I generates $N$ initial solutions from diverse prompts, and Stage II merges, evaluates, and potentially reinvents components via a structured fusion prompt. Extensive experiments across GSM8K, ALFWorld, Natural Plan, and TravelPlanner with Gemini-1.5 and GPT-4 show that SMaRT consistently surpasses traditional baselines including LLM-as-a-Judge and Self-Consistency, often achieving state-of-the-art results. The findings demonstrate the practical value of cross-strategy calibration for robust, efficient LLM-driven reasoning and planning, and point to future directions in memory-enabled and confidence-weighted fusion approaches.

Abstract

Large Language Models (LLMs) have redefined complex task automation with exceptional generalization capabilities. Despite these advancements, state-of-the-art methods rely on single-strategy prompting, missing the synergy of diverse reasoning approaches. No single strategy excels universally, highlighting the need for frameworks that fuse strategies to maximize performance and ensure robustness. We introduce the Select, Mix, and ReinvenT (SMaRT) framework, an innovative strategy fusion approach designed to overcome this constraint by creating balanced and efficient solutions through the seamless integration of diverse reasoning strategies. Unlike existing methods, which employ LLMs merely as evaluators, SMaRT uses them as intelligent integrators, unlocking the "best of all worlds" across tasks. Extensive empirical evaluations across benchmarks in reasoning, planning, and sequential decision-making highlight the robustness and adaptability of SMaRT. The framework consistently outperforms state-of-the-art baselines in solution quality, constraint adherence, and performance metrics. This work redefines LLM-driven decision-making by pioneering a new paradigm in cross-strategy calibration, unlocking superior outcomes for reasoning systems and advancing the boundaries of self-refining methodologies.

Figures (5)

• Figure 1: Complex real-world problems often involve multiple constraints that must be satisfied to generate a valid solution. For example, TravelPlanner xie2024travelplanner can involve up to thirteen constraints, from choosing a reasonable city route to a budget-friendly trip. As illustrated in the radar plot, no single prompting technique can effectively optimize all these constraints simultaneously.
• Figure 2: The proposed framework Select, Mix, and ReinvenT (SMaRT) for strategy fusion. To solve a task, the language model first employs base strategies to generate initial solutions based on the input information. In the second stage, the model has the chance to rethink the initial approaches and to develop a new solution. This new solution may incorporate elements from various initial solutions or introduce new components compared to the LLM-as-a-Judge or Self-Consistency framework that only chooses one answer.
• Figure 3: (left) The donut plot displays the percentage of time that the LLM-as-a-Judge selects each strategy. (right) The Venn diagram shows how SMaRT selects components from various base plans for the travel planner benchmark and demonstrates its ability to provide new solutions when existing options fall short. For more details, refer to appendix \ref{['app:reasoning']}.
• Figure 4: The performance of ALFWorld using GPT-4, highlighting the number of turns taken during the fusion phase of SMaRT and a successful Reflexion trial. The average steps needed to complete both individual sub-tasks and the overall task were analyzed and plotted for the six distinct task categories within this environment.
• Figure 5: Radar plots illustrate the performance of SMaRT and LLM-as-a-Judge across various dimensions of the travel planner. SMaRT demonstrates better overall performance.