VarParser: Unleashing the Neglected Power of Variables for LLM-based Log Parsing

TL;DR

VarParser tackles the inefficiencies and visibility gaps of constant-centric LLM-based log parsing by introducing a variable-centric framework that emphasizes variable parts of logs. It combines Variable Contribution Sampling, a Variable-Centric Parsing Cache with Variable Units, and Adaptive Variable-aware In-context Learning to preserve and utilize variable information, reducing LLM invocations and token costs while boosting accuracy. Across Loghub-2.0 datasets, VarParser achieves higher GA/PA/FTA and demonstrates substantial efficiency gains and cost reductions, with ablations confirming the importance of each module. The approach enhances system observability by retaining variable types and occurrences, enabling engineers to gain deeper insights for failure diagnosis and diagnosis tasks. The work suggests promising future directions in merging LLMs with smaller models to further optimize performance.

Abstract

Logs serve as a primary source of information for engineers to diagnose failures in large-scale online service systems. Log parsing, which extracts structured events from massive unstructured log data, is a critical first step for downstream tasks like anomaly detection and failure diagnosis. With advances in large language models (LLMs), leveraging their strong text understanding capabilities has proven effective for accurate log parsing. However, existing LLM-based log parsers all focus on the constant part of logs, ignoring the potential contribution of the variable part to log parsing. This constant-centric strategy brings four key problems. First, inefficient log grouping and sampling with only constant information. Second, a relatively large number of LLM invocations due to constant-based cache, leading to low log parsing accuracy and efficiency. Third, a relatively large number of consumed constant tokens in prompts leads to high LLM invocation costs. At last, these methods only retain placeholders in the results, losing the system visibility brought by variable information in logs. Facing these problems, we propose a variable-centric log parsing strategy named VarParser. Through variable contribution sampling, variable-centric parsing cache, and adaptive variable-aware in-context learning, our approach can efficiently capture the variable parts of logs and leverage their contributions to parsing. By introducing variable units, we preserve rich variable information, enhancing the integrity of log parsing results. Extensive evaluations on large-scale datasets demonstrate that VarParser achieves higher accuracy compared to existing methods, significantly improving parsing efficiency while reducing the LLM invocation costs.

Figures (9)

• Figure 1: An example of log parsing procedure. The red part of raw logs represents timestamps, the brown part represents the host and process information, the blue part represents the log level, and the green part represents the log content.
• Figure 2: The overall framework of VarParser. The demonstration variable set is initially extracted from historical logs through dynamic contribution sampling. The incoming log stream is matched against the cache. When matching fails, uncommon tokens are leveraged to select demonstration variables for LLM prompting.
• Figure 3: The workflow of variable contribution sampling, where both variable contribution and log contribution are dynamically updated during sampling.
• Figure 4: An illustrative case of variable omission in initial parsing, resulting in token matching failure for the log of the same template.
• Figure 5: An illustrative case comparing example selection methods. In token-level selection, examples are extracted from historical logs of different templates (L1).