RAFFLES: Reasoning-based Attribution of Faults for LLM Systems

TL;DR

RAFFLES introduces a reasoning-based, offline evaluation architecture for long-horizon LLM systems by coupling a Judge with multiple Evaluators in an iterative loop to identify decisive faults in agentic trajectories. It formalizes step- and decisive-fault notions and demonstrates that iterative, structured reasoning yields substantial performance gains over strong baselines on the Who&When and ReasonEval datasets, across diverse model families. The approach reduces manual debugging burden and shows versatility by extending to reasoning-chain fault attribution, signaling a scalable path toward automated, high-fidelity evaluation of complex multi-component systems. Limitations include data scarcity and the need for broader benchmarks, motivating future work on larger datasets and potential integration with reward-models for efficiency.

Abstract

The advent of complex, interconnected long-horizon LLM systems has made it incredibly tricky to identify where and when these systems break down. Evaluation capabilities that currently exist today are limited in that they often focus on simple metrics, end-to-end outcomes, and are dependent on the perspectives of humans. In order to match the increasing complexity of these many component systems, evaluation frameworks must also be able to reason, probe, iterate, and understand the nuanced logic passing through these systems. In this paper, we present RAFFLES, an offline evaluation architecture that incorporates iterative reasoning. Specifically, RAFFLES operates as an iterative, multi-component pipeline, using a central Judge to systematically identify faults and a set of specialized Evaluators to assess the quality of the candidate faults as well as rationales of the Judge. We evaluated RAFFLES with several benchmarks - the Who&When dataset to identify step-level faults in multi-agent systems and the ReasonEval datasets to diagnose step-level mathematical reasoning errors. RAFFLES outperforms strong baselines, achieving an accuracy of over 20% and 50% on the Who&When Hand-Crafted and Algorithmically-Generated datasets, and over 80% on the ReasonEval datasets. These results demonstrate a key step towards introducing automated fault detection for autonomous systems over labor-intensive manual review.

Figures (6)

• Figure 1: Our proposed RAFFLES framework for multi-turn agentic evaluation by reasoning-based fault attribution. RAFFLES leverages specialized Evaluators designed to assess candidate faults based on the criteria of a decisive fault. Each Evaluator takes in the full log $\tau$ and intermediate reasoning, which are passed to subsequent iterations until a decisive fault is determined.
• Figure 2: Llama 3.3 70B performance degrades with input trajectory length. Quartile of token length is with respect to the corresponding dataset.
• Figure 3: Llama 3.3 70B step-level accuracy vs. number of reasoning iterations
• Figure 4: Histogram of predicted and ground truth step for Who&When dataset, from iteration 1 to 3 for Llama 3.3 70B model. Notice that, for a given iteration $k$, we do not show the data points where convergence already happened in prior steps $k'<k$, hence the difference in ground truth histogram.
• Figure 5: Example log from the Who&When dataset, and how the RAFFLES iterative reasoning process achieves the correct decisive fault.

Theorems & Definitions (5)

• Definition 3.1: Step-Level Fault
• Definition 3.2: First Step-Level Fault Attribution
• Definition 3.3: Trivial Fault
• Definition 3.4: Critical Fault
• Definition 3.5: Decisive Fault Attribution