BugBlitz-AI: An Intelligent QA Assistant
Yi Yao, Jun Wang, Yabai Hu, Lifeng Wang, Yi Zhou, Jack Chen, Xuming Gai, Zhenming Wang, Wenjun Liu
TL;DR
BugBlitz-AI addresses the persistent post-execution bottleneck in automated software testing by automating result analysis and bug reporting with an AI-powered toolkit. The architecture combines a Service, Data Ingestion, and Action module with an Intelligent Analysis core that contains four LLM-based sub-modules for Root Error Analysis, Bug Diagnosis, Bug Summarization, and Duplicate Detection, guided by task decoupling, prompt engineering, and LoRA-based fine-tuning. The paper demonstrates recall and precision improvements through these quality-enhancement strategies, reporting strong recall and competitive precision on production-like data. Real-world results on production logs show BugBlitz-AI can autonomously generate Jira tickets and notifications, reducing manual effort and speeding up the software quality assurance lifecycle.
Abstract
The evolution of software testing from manual to automated methods has significantly influenced quality assurance (QA) practices. However, challenges persist in post-execution phases, particularly in result analysis and reporting. Traditional post-execution validation phases require manual intervention for result analysis and report generation, leading to inefficiencies and potential development cycle delays. This paper introduces BugBlitz-AI, an AI-powered validation toolkit designed to enhance end-to-end test automation by automating result analysis and bug reporting processes. BugBlitz-AI leverages recent advancements in artificial intelligence to reduce the time-intensive tasks of manual result analysis and report generation, allowing QA teams to focus more on crucial aspects of product quality. By adopting BugBlitz-AI, organizations can advance automated testing practices and integrate AI into QA processes, ensuring higher product quality and faster time-to-market. The paper outlines BugBlitz-AI's architecture, discusses related work, details its quality enhancement strategies, and presents results demonstrating its effectiveness in real-world scenarios.