LogSieve: Task-Aware CI Log Reduction for Sustainable LLM-Based Analysis

TL;DR

CI logs are essential but increasingly voluminous, incurring high inference costs and environmental impact when analyzed with LLMs. LogSieve introduces RCA-aware, semantics-preserving log reduction performed before inference to drop low-information lines while retaining content needed for root-cause analysis and failure categorization. On 20 Android GitHub Actions logs, LogSieve achieves about 42% line and 40% token reduction with minimal semantic loss, and downstream LLM tasks (explanation and categorization) maintain high fidelity (Cosine ~0.93; 80% exact-match) despite token reductions. Embedding-based classifiers can automate relevance detection with near-human accuracy, and LogSieve outperforms structure-first baselines like LogZip, enabling greener, interpretable CI automation without modifying model weights or prompts.

Abstract

Logs are essential for understanding Continuous Integration (CI) behavior, particularly for diagnosing build failures and performance regressions. Yet their growing volume and verbosity make both manual inspection and automated analysis increasingly costly, time-consuming, and environmentally costly. While prior work has explored log compression, anomaly detection, and LLM-based log analysis, most efforts target structured system logs rather than the unstructured, noisy, and verbose logs typical of CI workflows. We present LogSieve, a lightweight, RCA-aware and semantics-preserving log reduction technique that filters low-information lines while retaining content relevant to downstream reasoning. Evaluated on CI logs from 20 open-source Android projects using GitHub Actions, LogSieve achieves an average 42% reduction in lines and 40% reduction in tokens with minimal semantic loss. This pre-inference reduction lowers computational cost and can proportionally reduce energy use (and associated emissions) by decreasing the volume of data processed during LLM inference. Compared with structure-first baselines (LogZip and random-line removal), LogSieve preserves much higher semantic and categorical fidelity (Cosine = 0.93, GPTScore = 0.93, 80% exact-match accuracy). Embedding-based classifiers automate relevance detection with near-human accuracy (97%), enabling scalable and sustainable integration of semantics-aware filtering into CI workflows. LogSieve thus bridges log management and LLM reasoning, offering a practical path toward greener and more interpretable CI automation.

Figures (3)

• Figure 1: Examples of LogSieve reductions on Android CI log lines. RCA relevance: red = irrelevant, blue = relevant.
• Figure 2: An overview of LogSieve evaluation.
• Figure 3: An overview of automating LogSieve.