Identifying Concurrency Bug Reports via Linguistic Patterns

TL;DR

This work tackles the challenge of automatically identifying concurrency bug reports by introducing linguistic patterns (LPs) that describe concurrency phenomena at word, phrase, sentence, and bug-report levels. It derives 58 LPs from 730 labeled reports and evaluates four classification strategies, with LP-enriched fine-tuned pre-trained language models (notably CodeBERT) achieving the best performance (F1 up to 0.93) across GitHub, Jira, and post-cutoff data. Key contributions include the LP taxonomy, a novel LP-guided fine-tuning approach (CTagger), and a labeled dataset to support reproducibility. The approach improves precision and interpretability for concurrency bug classification and demonstrates strong generalization across projects and time, enabling scalable automation for issue triage and dataset creation.

Abstract

With the growing ubiquity of multi-core architectures, concurrent systems have become essential but increasingly prone to complex issues such as data races and deadlocks. While modern issue-tracking systems facilitate the reporting of such problems, labeling concurrency-related bug reports remains a labor-intensive and error-prone task. This paper presents a linguistic-pattern-based framework for automatically identifying concurrency bug reports. We derive 58 distinct linguistic patterns from 730 manually labeled concurrency bug reports, organized across four levels: word-level (keywords), phrase-level (n-grams), sentence-level (semantic), and bug report-level (contextual). To assess their effectiveness, we evaluate four complementary approaches-matching, learning, prompt-based, and fine-tuning-spanning traditional machine learning, large language models (LLMs), and pre-trained language models (PLMs). Our comprehensive evaluation on 12 large-scale open-source projects (10,920 issue reports from GitHub and Jira) demonstrates that fine-tuning PLMs with linguistic-pattern-enriched inputs achieves the best performance, reaching a precision of 91% on GitHub and 93% on Jira, and maintaining strong precision on post cut-off data (91%). The contributions of this work include: (1) a comprehensive taxonomy of linguistic patterns for concurrency bugs, (2) a novel fine-tuning strategy that integrates domain-specific linguistic knowledge into PLMs, and (3) a curated, labeled dataset to support reproducible research. Together, these advances provide a foundation for improving the automation, precision, and interpretability of concurrency bug classification.

Figures (24)

• Figure 1: False Negative of Keyword Search Method.
• Figure 2: Concurrency Related Question.
• Figure 3: Non-concurrency Issue.
• Figure 5: False Negative of Prompting GPT-4o.
• Figure 6: Overview of CTagger