ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization
Chunrong Fang, Weisong Sun, Yuchen Chen, Xiao Chen, Zhao Wei, Quanjun Zhang, Yudu You, Bin Luo, Yang Liu, Zhenyu Chen
TL;DR
Esale tackles code summarization by focusing on learning code–summary alignment through three summary-focused pre-training tasks: action word prediction (AWP), unidirectional language modeling (ULM), and masked language modeling (MLM). A pretrained encoder (e.g., UniXcoder) is fine-tuned with this multi-task objective, and a decoder is trained to generate natural language summaries, yielding improved results across four diverse datasets. Automatic metrics (BLEU, METEOR, ROUGE-L) and human judgments consistently favor Esale over strong baselines, with ablations showing all three tasks contribute, and AWP delivering the largest gains. The approach is compatible with existing pretrained models and offers a practical, non-intrusive path to better code summaries in real-world software engineering tasks.
Abstract
(Source) code summarization aims to automatically generate succinct natural language summaries for given code snippets. Such summaries play a significant role in promoting developers to understand and maintain code. Inspired by neural machine translation, deep learning-based code summarization techniques widely adopt an encoder-decoder framework, where the encoder transforms given code snippets into context vectors, and the decoder decodes context vectors into summaries. Recently, large-scale pre-trained models for source code are equipped with encoders capable of producing general context vectors and have achieved substantial improvements on code summarization. However, although they are usually trained mainly on code-focused tasks and can capture general code features, they still fall short in capturing specific features that need to be summarized. This paper proposes a novel approach to improve code summarization based on summary-focused tasks. Specifically, we exploit a multi-task learning paradigm to train the encoder on three summary-focused tasks to enhance its ability to learn code-summary alignment, including unidirectional language modeling (ULM), masked language modeling (MLM), and action word prediction (AWP). Unlike pre-trained models that mainly predict masked tokens in code snippets, we design ULM and MLM to predict masked words in summaries. Intuitively, predicting words based on given code snippets would help learn the code-summary alignment. Additionally, we introduce the domain-specific task AWP to enhance the ability of the encoder to learn the alignment between action words and code snippets. The extensive experiments on four datasets demonstrate that our approach, called ESALE significantly outperforms baselines in all three widely used metrics, including BLEU, METEOR, and ROUGE-L.