ReTrace: Interactive Visualizations for Reasoning Traces of Large Reasoning Models

TL;DR

ReTrace tackles the challenge of overly verbose large reasoning model traces by applying a validated reasoning taxonomy to structure traces and rendering them through two interactive visualizations: Space-Filling Nodes for hierarchical structure and Sequential Timeline for chronological flow. The system supports progressive disclosure, enabling detail-on-demand while preserving provenance of the original reasoning steps. In a within-subject user study with 18 participants, ReTrace led to improved comprehension of the model's strategy and reduced perceived workload compared with raw traces, with users showing greater ability to identify verification steps and judge the reasoning process. The work provides design implications for AI explainability that emphasize abstraction, structure, and active sensemaking, and suggests directions for real-time, richer visual representations of reasoning processes in future LRMs.

Abstract

Recent advances in Large Language Models have led to Large Reasoning Models, which produce step-by-step reasoning traces. These traces offer insight into how models think and their goals, improving explainability and helping users follow the logic, learn the process, and even debug errors. These traces, however, are often verbose and complex, making them cognitively demanding to comprehend. We address this challenge with ReTrace, an interactive system that structures and visualizes textual reasoning traces to support understanding. We use a validated reasoning taxonomy to produce structured reasoning data and investigate two types of interactive visualizations thereof. In a controlled user study, both visualizations enabled users to comprehend the model's reasoning more accurately and with less perceived effort than a raw text baseline. The results of this study could have design implications for making long and complex machine-generated reasoning processes more usable and transparent, an important step in AI explainability.

Figures (12)

• Figure 1: The three early prototypes explored in our exploratory study. From left to right: a Node-Link diagram emphasizing relational flow, Space-Filling Nodes emphasizing hierarchical structure, and a Sequential Timeline emphasizing chronological order.
• Figure 2: Example of a structured reasoning trace on a GSM8K sample, annotated by ReTrace. Colored annotations show LLM-generated summaries, indentation groups, and labels' reasoning taxonomy. Question from GSM8K bottom left, LRM generated answer bottom right. Legend shows assignment to the respective reasoning phase.
• Figure 3: The Space-Filling Nodes visualization in the ReTrace interface. This view organizes the reasoning trace hierarchically. The main panels represent primary reasoning phases, each showing (A) a summary, (B) its category label, and (C) the count of subphases and steps. Clicking a main panel reveals (D) nodes for each subphase, which can be further expanded to show (E) the raw text of individual reasoning steps.
• Figure 4: The Sequential Timeline visualization in the ReTrace interface. This view presents the reasoning trace chronologically. Each phase is represented by a box containing (A) a clickable summary and (B) its category label. The timeline bar below (C) shows the count of subphases and steps in proportional length of each reasoning phase. Expanding a phase reveals (D) nodes for subphases, which can be expanded to show (E) the raw text of individual steps. The category label at the bottom (F) shows the share of steps for the main phases.
• Figure 5: The architecture of the ReTrace data processing pipeline. The pipeline consists of three modules: (M1) Separator, which indexes the raw reasoning trace; (M2) Annotator, which uses an external LLM and a reasoning taxonomy to structure, summarize, and label the trace; and (M3) Visualizer, which generates the interactive web components for the two visualization types.