Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction

TL;DR

This work addresses the instability of intrinsic and externally guided iterative reflection in Large Language Models by proposing Instruct-of-Reflection (IoRT), a dynamic framework that uses a meta-thought–driven instructor to steer each reflection via refresh, stop, and select instructions. IoRT integrates a meta-thinker to generate high-level reasoning templates (meta-thoughts) and a self-consistency classifier to monitor iteration quality, enabling adaptive guidance across multiple iterations without oracle labels. Across mathematical and commonsense reasoning benchmarks (e.g., GSM8K, SVAMP, StrategyQA) and multiple model families, IoRT yields consistent accuracy gains and notable reductions in API calls and token usage, outperforming strong baselines such as CoT, PoT, and CRITIC. The approach demonstrates the practical value of dynamic meta-instruction for robust, efficient iterative reasoning in LLMs, with potential for broader applicability and further improvements via enhanced open-source reasoning modules.

Abstract

Self-reflection for Large Language Models (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs' internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce Instruct-of-Reflection (IoRT), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability.

Figures (5)

• Figure 1: Prior reflection research is primarily based on static iterative strategies, which face issues of redundancy, drift, and stubborn.
• Figure 2: Compare the performance changes of self-correct and CRITIC during iterative reflection across two datasets, evaluating with and without Oracle.
• Figure 3: Analyze the different types of iteration of CRITIC on GSM8K.
• Figure 4: IoRT comprises three steps: (1) generate meta-thoughts, (2) refresh and self-reflect, and (3) instruct the iteration. Repeat steps ⑧ to ①4 after each iteration until the instructor signals to stop or the maximum iterations $N$ is reached.
• Figure 5: Left: Comparison of the average number of API/LLM calls across various datasets on IoRT. Right: The performance of GPT and Llama-2 models on GSM8K with IoRT.