CoDesc: A Large Code-Description Parallel Dataset

TL;DR

CoDesc addresses the scarcity of large, clean code-description corpora for Java by compiling 4.2 million method-description pairs and applying systematic noise cleaning. The authors evaluate CoDesc on code summarization and code search, demonstrating up to 22% improvement in search and achieving state-of-the-art results for summarization. They also show that CoDesc is effective for pre-training and subsequent fine-tuning of Java-oriented language models, enabling pretrained models for code understanding. The work provides dataset, tooling, and benchmarks to facilitate reproducibility and further research in natural language to code and vice versa. Overall, CoDesc establishes a large-scale, cleaner resource to advance NL-to-code tasks and model pre-training.

Abstract

Translation between natural language and source code can help software development by enabling developers to comprehend, ideate, search, and write computer programs in natural language. Despite growing interest from the industry and the research community, this task is often difficult due to the lack of large standard datasets suitable for training deep neural models, standard noise removal methods, and evaluation benchmarks. This leaves researchers to collect new small-scale datasets, resulting in inconsistencies across published works. In this study, we present CoDesc -- a large parallel dataset composed of 4.2 million Java methods and natural language descriptions. With extensive analysis, we identify and remove prevailing noise patterns from the dataset. We demonstrate the proficiency of CoDesc in two complementary tasks for code-description pairs: code summarization and code search. We show that the dataset helps improve code search by up to 22\% and achieves the new state-of-the-art in code summarization. Furthermore, we show CoDesc's effectiveness in pre-training--fine-tuning setup, opening possibilities in building pretrained language models for Java. To facilitate future research, we release the dataset, a data processing tool, and a benchmark at \url{https://github.com/csebuetnlp/CoDesc}.