Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs

Alireza Ezaz, Ghazal Khodabandeh, Majid Babaei, Naser Ezzati-Jivan

TL;DR

TAAF tackles the challenge of analyzing massive kernel traces by introducing a three-layer framework that first abstracts traces into a time-indexed state system, then grounds query-specific semantics in a knowledge graph, and finally leverages LLMs to generate grounded natural-language answers. The approach is evaluated with TraceQA-100, a dedicated benchmark of kernel-trace QA tasks, showing substantial accuracy gains (up to 31.2%) across multiple LLM backends and temporal settings, especially for multi-hop and causal reasoning. The study demonstrates that graph-grounded reasoning improves robustness and explainability over raw or flattened inputs, and discusses how temporal grounding, schema usage, and model choice influence performance. The work provides a foundation for scalable, explainable trace analysis tools and outlines future directions in temporal KG reasoning and integration with autonomous workflows.

Abstract

Execution traces are a critical source of information for understanding, debugging, and optimizing complex software systems. However, traces from OS kernels or large-scale applications like Chrome or MySQL are massive and difficult to analyze. Existing tools rely on predefined analyses, and custom insights often require writing domain-specific scripts, which is an error-prone and time-consuming task. This paper introduces TAAF (Trace Abstraction and Analysis Framework), a novel approach that combines time-indexing, knowledge graphs (KGs), and large language models (LLMs) to transform raw trace data into actionable insights. TAAF constructs a time-indexed KG from trace events to capture relationships among entities such as threads, CPUs, and system resources. An LLM then interprets query-specific subgraphs to answer natural-language questions, reducing the need for manual inspection and deep system expertise. To evaluate TAAF, we introduce TraceQA-100, a benchmark of 100 questions grounded in real kernel traces. Experiments across three LLMs and multiple temporal settings show that TAAF improves answer accuracy by up to 31.2%, particularly in multi-hop and causal reasoning tasks. We further analyze where graph-grounded reasoning helps and where limitations remain, offering a foundation for next-generation trace analysis tools.

TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs

TL;DR

TAAF tackles the challenge of analyzing massive kernel traces by introducing a three-layer framework that first abstracts traces into a time-indexed state system, then grounds query-specific semantics in a knowledge graph, and finally leverages LLMs to generate grounded natural-language answers. The approach is evaluated with TraceQA-100, a dedicated benchmark of kernel-trace QA tasks, showing substantial accuracy gains (up to 31.2%) across multiple LLM backends and temporal settings, especially for multi-hop and causal reasoning. The study demonstrates that graph-grounded reasoning improves robustness and explainability over raw or flattened inputs, and discusses how temporal grounding, schema usage, and model choice influence performance. The work provides a foundation for scalable, explainable trace analysis tools and outlines future directions in temporal KG reasoning and integration with autonomous workflows.

Abstract

Execution traces are a critical source of information for understanding, debugging, and optimizing complex software systems. However, traces from OS kernels or large-scale applications like Chrome or MySQL are massive and difficult to analyze. Existing tools rely on predefined analyses, and custom insights often require writing domain-specific scripts, which is an error-prone and time-consuming task. This paper introduces TAAF (Trace Abstraction and Analysis Framework), a novel approach that combines time-indexing, knowledge graphs (KGs), and large language models (LLMs) to transform raw trace data into actionable insights. TAAF constructs a time-indexed KG from trace events to capture relationships among entities such as threads, CPUs, and system resources. An LLM then interprets query-specific subgraphs to answer natural-language questions, reducing the need for manual inspection and deep system expertise. To evaluate TAAF, we introduce TraceQA-100, a benchmark of 100 questions grounded in real kernel traces. Experiments across three LLMs and multiple temporal settings show that TAAF improves answer accuracy by up to 31.2%, particularly in multi-hop and causal reasoning tasks. We further analyze where graph-grounded reasoning helps and where limitations remain, offering a foundation for next-generation trace analysis tools.
Paper Structure (26 sections, 9 equations, 9 figures, 4 tables)

This paper contains 26 sections, 9 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Trace Abstraction and Stateful Modeling
  4. Knowledge Graphs for Execution Context Modeling
  5. LLM–KG Integration Strategies
  6. Methodology
  7. Problem Setting and Motivation
  8. Architectural Overview and Trace-to-Answer Pipeline
  9. Layer 1: From Raw Trace to State System Abstraction
  10. Layer 2: From State System to Query-Specific Knowledge Graph
  11. Layer 3: From Knowledge Graph to Answer via LLMs
  12. Empirical Evaluation
  13. Evaluation Framework: TraceQA-100
  14. Evaluation Setup and Metrics
  15. Model Configuration.
  16. ...and 11 more sections

Figures (9)

  • Figure 1: End-to-end architecture overview of the Trace Abstraction and Analysis Framework (TAAF).
  • Figure 2: Example of a query-specific knowledge graph generated from a 1-second trace interval.
  • Figure 3: Accuracy by query type and hop count. Left: baseline; right: TAAF. White text shows accuracy gain (+p.p.).
  • Figure 4: Accuracy at for each model and interval
  • Figure 5: Accuracy gain from KG in TAAF .
  • ...and 4 more figures