Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering

Durga Prasad Maram, Kalpa Gunaratna, Vijay Srinivasan, Haris Jeelani, Srinivas Chappidi

TL;DR

This work tackles multi-hop QA by generating diverse reasoning traces with Monte Carlo Tree Search and refining them through sub-answer verification and LLM-based judgments. The resulting high-quality traces are used to fine-tune LLMs to reformulate sub-queries when retrieval fails, reducing hallucinations and improving retrieval-grounded correctness. Experiments across four datasets show PATHFINDER often outperforms DeepRAG baselines, with the LLM Judge filtering contributing the largest gains. The approach highlights the importance of grounded reasoning traces and retrieval-aware training for robust multi-hop question answering.

Abstract

Multi-hop question answering is a challenging task in which language models must reason over multiple steps to reach the correct answer. With the help of Large Language Models and their reasoning capabilities, existing systems are able to think and decompose an input question over multiple steps to analyze, retrieve, and reason. However, training-based approaches for this problem still suffer from LLM hallucinations and incorrect reasoning paths that hinder performance. Hence, we propose PATHFINDER, an approach that: (i) uses Monte Carlo Tree Search to generate training path traces, (ii) improves training data quality by filtering erroneous and lengthy traces using sub-answer recall and LLM-as-a-judge verification, and (iii) reformulates sub-queries to handle failed retrieval cases. By following these steps, we demonstrate that PATHFINDER improves the performance of multi-hop QA over public benchmark datasets.

PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering

TL;DR

This work tackles multi-hop QA by generating diverse reasoning traces with Monte Carlo Tree Search and refining them through sub-answer verification and LLM-based judgments. The resulting high-quality traces are used to fine-tune LLMs to reformulate sub-queries when retrieval fails, reducing hallucinations and improving retrieval-grounded correctness. Experiments across four datasets show PATHFINDER often outperforms DeepRAG baselines, with the LLM Judge filtering contributing the largest gains. The approach highlights the importance of grounded reasoning traces and retrieval-aware training for robust multi-hop question answering.

Abstract

Multi-hop question answering is a challenging task in which language models must reason over multiple steps to reach the correct answer. With the help of Large Language Models and their reasoning capabilities, existing systems are able to think and decompose an input question over multiple steps to analyze, retrieve, and reason. However, training-based approaches for this problem still suffer from LLM hallucinations and incorrect reasoning paths that hinder performance. Hence, we propose PATHFINDER, an approach that: (i) uses Monte Carlo Tree Search to generate training path traces, (ii) improves training data quality by filtering erroneous and lengthy traces using sub-answer recall and LLM-as-a-judge verification, and (iii) reformulates sub-queries to handle failed retrieval cases. By following these steps, we demonstrate that PATHFINDER improves the performance of multi-hop QA over public benchmark datasets.

Paper Structure

This paper contains 9 sections, 2 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Approach
  4. Answer Trace Generation
  5. Filtering Answer Traces
  6. Training
  7. Evaluation
  8. Results & Discussion
  9. Conclusion

Figures (3)

  • Figure 1: Design pipeline of PATHFINDER. It involves (i) CoT trace generation with MCTS, (ii) filtering CoT traces with sub-answer verification, (iii) filtering CoT traces with LLM-as-a-Judge, and (iv) training the target LLM with optimal traces based on the LLM Judge scores and the trace length. Intermediate OST and few context steps are omitted above. The LLM Judge validation example demonstrates one of the 4 criteria we consider, the faithfulness of sub-answer generation.
  • Figure 2: DeepRAG (Imi-retrieve-only) vs. PATHFINDER on F1 score across different question types in 2WikiMultiHopQA.
  • Figure 3: PATHFINDER vs DeepRAG. PATHFINDER finds the right answer through effective query reformulation (highlighted sub-questions in purple) when relevant context is not retrieved.