FutureMind: Equipping Small Language Models with Strategic Thinking-Pattern Priors via Adaptive Knowledge Distillation

TL;DR

FutureMind tackles the challenge that small language models struggle with knowledge-intensive, multi-hop reasoning by transferring structured thinking patterns from large language models through adaptive knowledge distillation. It introduces a four-stage modular pipeline (Problem Analysis, Logical Reasoning, Strategy Planning, Retrieval Guidance) and three composable retrieval paradigms to balance reasoning depth with efficiency, achieving strong, training-free improvements across four multi-hop QA benchmarks. Key findings include consistent gains for small models, evidence of a cognitive bias bottleneck in teacher–student distillation, and the importance of architecture–alignment between teacher and student. The work demonstrates that equipping SLMs with strategic thinking priors can yield scalable, interpretable retrieval-based reasoning suitable for resource-constrained settings with broad practical impact.

Abstract

Small Language Models (SLMs) are attractive for cost-sensitive and resource-limited settings due to their efficient, low-latency inference. However, they often struggle with complex, knowledge-intensive tasks that require structured reasoning and effective retrieval. To address these limitations, we propose FutureMind, a modular reasoning framework that equips SLMs with strategic thinking-pattern priors via adaptive knowledge distillation from large language models (LLMs). FutureMind introduces a dynamic reasoning pipeline composed of four key modules: Problem Analysis, Logical Reasoning, Strategy Planning, and Retrieval Guidance. This pipeline is augmented by three distinct retrieval paradigms that decompose complex queries into tractable subproblems, ensuring efficient and accurate retrieval execution. Extensive experiments on multi-hop QA benchmarks, including 2WikiMultihopQA, MuSiQue, Bamboogle, and Frames, demonstrate the superiority of FutureMind. It consistently outperforms strong baselines such as Search-o1, achieving state-of-the-art results under free training conditions across diverse SLM architectures and scales. Beyond empirical gains, our analysis reveals that the process of thinking-pattern distillation is restricted by the cognitive bias bottleneck between the teacher (LLMs) and student (SLMs) models. This provides new perspectives on the transferability of reasoning skills, paving the way for the development of SLMs that combine efficiency with genuine cognitive capability.

Figures (4)

• Figure 1: Overall performance comparison of FutureMind with other methods across four multi-hop QA benchmarks. The left panel depicts the performance on a 3B small language model (SLM), while the right panel illustrates the performance on a 72B large language model (LLM).
• Figure 2: Overview of the FutureMind framework.
• Figure 3: Ablation study of enhanced ToolCall across different models.
• Figure 4: Three adaptive retrieval paradigms employed in FutureMind.