Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LUNAR: Unsupervised LLM-based Log Parsing

Junjie Huang, Zhihan Jiang, Zhuangbin Chen, Michael R. Lyu

TL;DR

LUNAR introduces a label-free, LLM-based approach to log parsing that leverages Log Contrastive Units (LCUs) and a hierarchical sharding strategy to enable scalable, unsupervised extraction of log templates and parameters. It combines a two-stage LCU selector with a carefully designed, demonstration-free prompt to guide zero-shot parsing, achieving state-of-the-art performance among unsupervised parsers and competitive results with supervised methods on the Loghub-2.0 benchmark. The framework emphasizes parallelization and efficiency, with experiments showing substantial speedups in parallel mode and robust accuracy across diverse datasets. Overall, LUNAR demonstrates that unsupervised, contrastive analysis with LLMs can robustly parse evolving, large-scale logs without labeled data, supporting practical deployment in production systems.

Abstract

Log parsing serves as an essential prerequisite for various log analysis tasks. Recent advancements in this field have improved parsing accuracy by leveraging the semantics in logs through fine-tuning large language models (LLMs) or learning from in-context demonstrations. However, these methods heavily depend on labeled examples to achieve optimal performance. In practice, collecting sufficient labeled data is challenging due to the large scale and continuous evolution of logs, leading to performance degradation of existing log parsers after deployment. To address this issue, we propose LUNAR, an unsupervised LLM-based method for efficient and off-the-shelf log parsing. Our key insight is that while LLMs may struggle with direct log parsing, their performance can be significantly enhanced through comparative analysis across multiple logs that differ only in their parameter parts. We refer to such groups of logs as Log Contrastive Units (LCUs). Given the vast volume of logs, obtaining LCUs is difficult. Therefore, LUNAR introduces a hybrid ranking scheme to effectively search for LCUs by jointly considering the commonality and variability among logs. Additionally, LUNAR crafts a novel parsing prompt for LLMs to identify contrastive patterns and extract meaningful log structures from LCUs. Experiments on large-scale public datasets demonstrate that LUNAR significantly outperforms state-of-the-art log parsers in terms of accuracy and efficiency, providing an effective and scalable solution for real-world deployment. \footnote{The code and data are available at \url{https://github.com/Jun-jie-Huang/LUNAR}}.

LUNAR: Unsupervised LLM-based Log Parsing

TL;DR

LUNAR introduces a label-free, LLM-based approach to log parsing that leverages Log Contrastive Units (LCUs) and a hierarchical sharding strategy to enable scalable, unsupervised extraction of log templates and parameters. It combines a two-stage LCU selector with a carefully designed, demonstration-free prompt to guide zero-shot parsing, achieving state-of-the-art performance among unsupervised parsers and competitive results with supervised methods on the Loghub-2.0 benchmark. The framework emphasizes parallelization and efficiency, with experiments showing substantial speedups in parallel mode and robust accuracy across diverse datasets. Overall, LUNAR demonstrates that unsupervised, contrastive analysis with LLMs can robustly parse evolving, large-scale logs without labeled data, supporting practical deployment in production systems.

Abstract

Log parsing serves as an essential prerequisite for various log analysis tasks. Recent advancements in this field have improved parsing accuracy by leveraging the semantics in logs through fine-tuning large language models (LLMs) or learning from in-context demonstrations. However, these methods heavily depend on labeled examples to achieve optimal performance. In practice, collecting sufficient labeled data is challenging due to the large scale and continuous evolution of logs, leading to performance degradation of existing log parsers after deployment. To address this issue, we propose LUNAR, an unsupervised LLM-based method for efficient and off-the-shelf log parsing. Our key insight is that while LLMs may struggle with direct log parsing, their performance can be significantly enhanced through comparative analysis across multiple logs that differ only in their parameter parts. We refer to such groups of logs as Log Contrastive Units (LCUs). Given the vast volume of logs, obtaining LCUs is difficult. Therefore, LUNAR introduces a hybrid ranking scheme to effectively search for LCUs by jointly considering the commonality and variability among logs. Additionally, LUNAR crafts a novel parsing prompt for LLMs to identify contrastive patterns and extract meaningful log structures from LCUs. Experiments on large-scale public datasets demonstrate that LUNAR significantly outperforms state-of-the-art log parsers in terms of accuracy and efficiency, providing an effective and scalable solution for real-world deployment. \footnote{The code and data are available at \url{https://github.com/Jun-jie-Huang/LUNAR}}.
Paper Structure (31 sections, 4 equations, 9 figures, 2 tables)

This paper contains 31 sections, 4 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Limitations of Semantic-based Log Parsers
  4. Log Contrastive Unit for LLM-based Parsing
  5. Methodology
  6. Overview
  7. Hierarchical Log Sharder
  8. Length-based Sharding.
  9. Top-$k$ Token Agglomerative Clustering.
  10. Two-stage LCU Selector
  11. Stratified LCU Generation.
  12. Hybrid LCU Ranking
  13. Label-free LLM Parser
  14. Prompt Design
  15. Template Acquisition
  16. ...and 16 more sections

Figures (9)

  • Figure 1: An example of log parsing procedure.
  • Figure 2: Empirical study on the influences of label proportion on semantic-based log parsers.
  • Figure 3: Examples of log contrastive units (LCUs).
  • Figure 4: The overall workflow of LUNAR.
  • Figure 5: Hierarchical log sharding in LUNAR.
  • ...and 4 more figures