Mirror: A Multiple-perspective Self-Reflection Method for Knowledge-rich Reasoning

TL;DR

This work addresses the difficulty of improving knowledge-rich reasoning when ground-truth signals are unavailable by introducing Mirror, a multi-perspective self-reflection framework that couples a Navigator with a Reasoner. Mirror uses diversity-based rewards and inter-consistency among strategically perturbed responses, guided by Monte-Carlo Tree Search, to explore multiple reasoning trajectories without ground truth. Empirical results on MMLU and FEVER show Mirror achieving substantial improvements (over 15% relative) over strong unsupervised self-refinement baselines, with ablations confirming the benefits of question-oriented directions and diverse search spaces. The approach reduces reliance on external ground truth and demonstrates a scalable pathway to more reliable knowledge-rich reasoning in LLMs. The work has practical implications for enhancing first-pass reasoning in AI systems across domains where ground-truth feedback is scarce or costly.

Abstract

While Large language models (LLMs) have the capability to iteratively reflect on their own outputs, recent studies have observed their struggles with knowledge-rich problems without access to external resources. In addition to the inefficiency of LLMs in self-assessment, we also observe that LLMs struggle to revisit their predictions despite receiving explicit negative feedback. Therefore, We propose Mirror, a Multiple-perspective self-reflection method for knowledge-rich reasoning, to avoid getting stuck at a particular reflection iteration. Mirror enables LLMs to reflect from multiple-perspective clues, achieved through a heuristic interaction between a Navigator and a Reasoner. It guides agents toward diverse yet plausibly reliable reasoning trajectory without access to ground truth by encouraging (1) diversity of directions generated by Navigator and (2) agreement among strategically induced perturbations in responses generated by the Reasoner. The experiments on five reasoning datasets demonstrate that Mirror's superiority over several contemporary self-reflection approaches. Additionally, the ablation study studies clearly indicate that our strategies alleviate the aforementioned challenges.

Figures (9)

• Figure 1: Without ground truth for validating LLM-generated outputs, LLMs struggle to consistently improve their own outputs due to their incapability of self-assessment. Autostop and Neverstop provide different generic feedback without leaking the correctness of the current response.
• Figure 2: The average number (across all iterations) of changed and unchanged samples among those predicted incorrectly. Large percentage of unchanged samples indicate the limited capability for efficient reflection.
• Figure 3: An overview of Mirror. It facilitates diverse question-specific directions (represented by different colored dots in the action space) to encourage extensive reflection by the Reasoner. The stopping criterion is based on the consistency among states from multiple perspectives, which also contributes to the direction generation.
• Figure 4: Reasoning process of self-correction and Mirror. Text in red are generated directions. Our diversity is characterised in (i) generating directions tailored to questions (ii) encouraging exploration in multiple plausible reasoning paths. The final answer is derived through an agreement among multiple trajectories.
• Figure 5: The Accuracy (acc) and the percentage of samples where the ground truth is included in the tree (ans-presence), with different sizes of search space (Num). Results for GPT-3.5 and Llama13B are in Figure \ref{['fig:acc_pres_other1']} and \ref{['fig:acc_pres_other2']}.