Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism

TL;DR

This paper addresses how chain-of-thought fine-tuning can be analyzed and advanced from a cognitive perspective by applying the Six Thinking Hats framework. It proposes a bi-level taxonomy that links hat-based reasoning modes with concrete CoT techniques, spanning Reflex-to-Thinking (SFT/RFT) transitions to Insight-model capabilities such as planning, divergent thinking, intuition, reflection, internalization, and factual grounding. It provides a comprehensive survey of data acquisition methods, reward modeling, and evaluation benchmarks, and offers forward-looking directions grounded in human reasoning theory. The work culminates in an open-source resource to track progress, aiming to guide researchers toward more holistic, robust, and trustworthy reasoning in large language models.

Abstract

Chain of thought (CoT) fine-tuning aims to endow large language models (LLMs) with reasoning capabilities by training them on curated reasoning traces. It leverages both supervised and reinforced fine-tuning to cultivate human-like reasoning skills in LLMs, including detailed planning, divergent thinking, intuitive judgment, timely reflection, internal thinking, and fact perception, etc. As CoT fine-tuning has advanced, LLMs have demonstrated substantial improvements in tasks such as mathematical reasoning and code generation. However, existing surveys about CoT fine-tuning primarily focus on technical aspects and overlook a systematic analysis from the perspective of human reasoning mechanisms. Given that the ultimate goal of CoT fine-tuning is to enable LLMs to reason like humans, it is crucial to investigate this technique through the lens of human cognition. To fill this gap, we present the first comprehensive survey of CoT fine-tuning grounded in human reasoning theory. Specifically, inspired by the well-known Six Thinking Hats framework, which systematically characterizes common human thinking modes using six metaphorical hats, we classify and examine CoT fine-tuning methods through this lens. Furthermore, building upon this theory, we outline potential directions for future research in CoT fine-tuning. In addition, we compile a comprehensive overview of existing datasets and model performances, and a real-time GitHub repository \footnote{https://github.com/AI-Chen/Awesome-CoT-Finetuning} that continuously tracks recent advances in this area is maintained. We hope this survey will serve as a valuable resource to inspire innovation and foster progress in this rapidly evolving field.

Figures (11)

• Figure 1: The evolution of CoT fine-tuning. It can be divided into three stages. 1) Reflex model stage, which focuses on training LLMs to produce direct answers; 2) Think Model stage, which enables step-by-step reasoning; and 3) Insight Model stage, which aims to equip LLMs with human-like thinking abilities. Drawing on the Six Thinking Hats framework, we categorize the development of CoT fine-tuning in the insight Model stage into six distinct dimensions: planning, divergent thinking, intuition, reflection, implicit reasoning, and factual perception. These capabilities correspond respectively to the six metaphorical hats: blue (management), green (creativity), red (intuition), black (criticism), yellow (optimism), and white (objectivity). Together, these hats are employed interactively to form a comprehensive and mature reasoning framework, as described in the Six Thinking Hats theory.
• Figure 2: Taxonomy of CoT fine-tuning. TTD, MAD, and f&s thinking refer to test time diversity, multi-agent debate, and the integration of slow and fast thinking, respectively.
• Figure 3: The training pipeline of converting LLMs from the Reflex Model to the Thinking Model. The primary objective of SFT is to train LLMs to generate both the CoT and the final answer given an input question, while that of RFT is to strengthen the reasoning ability of LLMs.
• Figure 4: The comparison among RFT methods.
• Figure 5: The framework of equipping LLMs with the planning ability.