Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ART: Adaptive Reasoning Trees for Explainable Claim Verification

Sahil Wadhwa, Himanshu Kumar, Guanqun Yang, Abbaas Alif Mohamed Nishar, Pranab Mohanty, Swapnil Shinde, Yue Wu

TL;DR

This paper proposes ART (Adaptive Reasoning Trees), a hierarchical method for claim verification which outperforms strong baselines, establishing a new benchmark for explainable claim verification which is more reliable and ensures clarity in the overall decision making step.

Abstract

Large Language Models (LLMs) are powerful candidates for complex decision-making, leveraging vast encoded knowledge and remarkable zero-shot abilities. However, their adoption in high-stakes environments is hindered by their opacity; their outputs lack faithful explanations and cannot be effectively contested to correct errors, undermining trustworthiness. In this paper, we propose ART (Adaptive Reasoning Trees), a hierarchical method for claim verification. The process begins with a root claim, which branches into supporting and attacking child arguments. An argument's strength is determined bottom-up via a pairwise tournament of its children, adjudicated by a judge LLM, allowing a final, transparent and contestable verdict to be systematically derived which is missing in methods like Chain-of-Thought (CoT). We empirically validate ART on multiple datasets, analyzing different argument generators and comparison strategies. Our findings show that ART's structured reasoning outperforms strong baselines, establishing a new benchmark for explainable claim verification which is more reliable and ensures clarity in the overall decision making step.

ART: Adaptive Reasoning Trees for Explainable Claim Verification

TL;DR

This paper proposes ART (Adaptive Reasoning Trees), a hierarchical method for claim verification which outperforms strong baselines, establishing a new benchmark for explainable claim verification which is more reliable and ensures clarity in the overall decision making step.

Abstract

Large Language Models (LLMs) are powerful candidates for complex decision-making, leveraging vast encoded knowledge and remarkable zero-shot abilities. However, their adoption in high-stakes environments is hindered by their opacity; their outputs lack faithful explanations and cannot be effectively contested to correct errors, undermining trustworthiness. In this paper, we propose ART (Adaptive Reasoning Trees), a hierarchical method for claim verification. The process begins with a root claim, which branches into supporting and attacking child arguments. An argument's strength is determined bottom-up via a pairwise tournament of its children, adjudicated by a judge LLM, allowing a final, transparent and contestable verdict to be systematically derived which is missing in methods like Chain-of-Thought (CoT). We empirically validate ART on multiple datasets, analyzing different argument generators and comparison strategies. Our findings show that ART's structured reasoning outperforms strong baselines, establishing a new benchmark for explainable claim verification which is more reliable and ensures clarity in the overall decision making step.
Paper Structure (40 sections, 4 theorems, 19 equations, 9 figures, 3 tables)

This paper contains 40 sections, 4 theorems, 19 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Explainable LLM-based Decision Making
  4. Automated Claim Verification
  5. Multi-Agent Debate
  6. Comparison with ArgLLMs
  7. Methodology
  8. Argument Generation
  9. Pairwise Argument Ranking
  10. Strength Calibration
  11. Bradley--Terry Calibration.
  12. Strength Aggregation
  13. Final Claim Score.
  14. Experiments
  15. Datasets
  16. ...and 25 more sections

Key Result

Proposition 1

For any $c>0$, replacing all cross-pair counts by $c\,\tau_{ab}$ (hence $n_{ab}\mapsto c\,n_{ab}$) leaves the mapping $\theta^{(t)}\mapsto\theta^{(t+1)}$ in Eq. eq:bt_update unchanged after normalization; thus the fixed point $\hat{\theta}$ (up to scale) is unchanged.

Figures (9)

  • Figure 1: Overview of the ART (Adaptive Reasoning Trees) framework with a tree of depth=$1$ and breadth=$3$ for illustration. Each breadth adds $b$ support and attack arguments. A claim is first processed by the Argument Generator, which creates a tree of supporting and attacking arguments. Arguments of opposing stances are then pitted against each other in a pairwise tournament. The Pairwise Argument Ranker, an LLM-as-a-judge, evaluates these pairs to dynamically update the strength of each argument. Finally, the Strength Aggregator consolidates these updated scores into a final probability for the claim's veracity.
  • Figure 2: Example from MedQA with Llama 3.3 70B Instruct. Pairwise tournaments calibrate argument strengths by blending intrinsic weights $\tau$ with BT scores $\theta$ to yield $\tau'$. Here, support and attack start with equal intrinsic weights but calibrate to $0.7$ and $0.4$, shifting the claim’s probability from $0.5$ (False) to $0.8$ (True). Without the pairwise and BT calibration steps, the claim would be misclassified as False.
  • Figure 3: Variance of strengths distribution under ArgLLM ($\lambda=0$) vs. ART ($\lambda=0.5$). of the root claim nodes. ART yields higher variance—scores pushed away from $0.5$—indicating more decisive, calibrated strengths.
  • Figure 4: Examples where ART improves the prediction results due to calibration that happens as a result of pairwise tournaments. Increasing the number of support and attack arguments in the tournament provides greater stability to the estimated strengths.
  • Figure 5: Direct prompt.
  • ...and 4 more figures

Theorems & Definitions (7)

  • Proposition 1: Scale invariance under bipartite comparisons
  • proof
  • Proposition 2: MM minorization and monotone ascent for $\varepsilon=0$ (bipartite)
  • proof
  • Proposition 3: DF-QuAD combination via signed product gap
  • proof
  • Proposition 4: Counts on a full $2b$-ary tree of depth $d$