Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey

TL;DR

This paper addresses the reliability gap in language models for code intelligence (LM4Code) by conducting a systematic literature review of 67 primary studies from 2018–2023. It develops a four‑part taxonomy of pitfalls aligned with the LM4Code lifecycle: data collection/ labeling, system design/learning, performance evaluation, and deployment/maintenance, and quantifies implications and remedies for each. Key contributions include a rigorous synthesis of data‑stage issues (unbalanced data, noise, labeling errors), design/learning pitfalls (data snooping, spurious correlations, poor model design), evaluation pitfalls (baselines, datasets, reproducibility, metrics), and deployment challenges (real‑world constraints, attacks, security of generated code), along with practical recommendations (data cleaning, robust benchmarks, interpretability, adversarial training, privacy protections). The work provides a roadmap for more robust, trustworthy LM4Code, emphasizing standardized evaluation, data quality assurance, and security‑aware deployment to enable reliable real‑world adoption in software engineering.

Abstract

Modern language models (LMs) have been successfully employed in source code generation and understanding, leading to a significant increase in research focused on learning-based code intelligence, such as automated bug repair, and test case generation. Despite their great potential, language models for code intelligence (LM4Code) are susceptible to potential pitfalls, which hinder realistic performance and further impact their reliability and applicability in real-world deployment. Such challenges drive the need for a comprehensive understanding - not just identifying these issues but delving into their possible implications and existing solutions to build more reliable language models tailored to code intelligence. Based on a well-defined systematic research approach, we conducted an extensive literature review to uncover the pitfalls inherent in LM4Code. Finally, 67 primary studies from top-tier venues have been identified. After carefully examining these studies, we designed a taxonomy of pitfalls in LM4Code research and conducted a systematic study to summarize the issues, implications, current solutions, and challenges of different pitfalls for LM4Code systems. We developed a comprehensive classification scheme that dissects pitfalls across four crucial aspects: data collection and labeling, system design and learning, performance evaluation, and deployment and maintenance. Through this study, we aim to provide a roadmap for researchers and practitioners, facilitating their understanding and utilization of LM4Code in reliable and trustworthy ways.

Figures (15)

• Figure 1: Distribution of papers over years
• Figure 2: Distribution of papers across LMs
• Figure 3: The overview of pitfalls of LMs for code intelligence
• Figure 4: Paper distribution across time (Section \ref{['sec:data']})
• Figure 5: Distribution of LMs (Section \ref{['sec:data']})