Optimizing Knowledge Utilization for Multi-Intent Comment Generation with Large Language Models

TL;DR

This work tackles multi-intent code comment generation by proposing KUMIC, a framework that optimizes knowledge utilization for LLMs through in-context learning and Chain-of-Thought. KUMIC comprises an Example Retriever that selects high-quality, intent-aligned demonstrations using token and semantic similarity plus an example-quality score, and Knowledge Augmentation that extracts intent-specific statements via a lightweight code-search model to form a mapping chain from code to statements to comments. The approach is evaluated on Funcom and TLC Java datasets across three LLMs (CodeLlama, Llama3, Qwen2.5-Coder), showing substantial improvements over state-of-the-art baselines in BLEU, METEOR, ROUGE-L, and SBERT, with strong human judgments on accuracy, adequacy, and intention alignment. The results demonstrate the practicality of augmenting demonstrations with intent-relevant knowledge and using a code-search–driven extraction to guide reasoning, particularly under data-scarce conditions, making multi-intent comment generation more reliable and actionable for developers.

Abstract

Code comment generation aims to produce a generic overview of a code snippet, helping developers understand and maintain code. However, generic summaries alone are insufficient to meet the diverse needs of practitioners; for example, developers expect the implementation insights to be presented in an untangled manner, while users seek clear usage instructions. This highlights the necessity of multi-intent comment generation. With the widespread adoption of Large Language Models (LLMs) for code-related tasks, these models have been leveraged to tackle the challenge of multi-intent comment generation. Despite their successes, state-of-the-art LLM-based approaches often struggle to construct correct relationships among intents, code, and comments within a smaller number of demonstration examples. To mitigate this issue, we propose a framework named KUMIC for multi-intent comment generation. Built upon in-context learning, KUMIC leverages Chain-of-Thought (CoT) to optimize knowledge utilization for LLMs to generate intent-specific comments. Specifically, KUMIC first designs a retrieval mechanism to obtain similar demonstration examples, which exhibit high code-comment consistency. Then, KUMIC leverages CoT to guide LLMs to focus on statements facilitating the derivation of code comments aligned with specific intents. In this context, KUMIC constructs a mapping knowledge chain, linking code to intent-specific statements to comments, which enables LLMs to follow similar reasoning steps when generating the desired comments. We conduct extensive experiments to evaluate KUMIC, and the results demonstrate that KUMIC outperforms state-of-the-art baselines by 14.49\%, 22.41\%, 20.72\%, and 12.94\% in terms of BLEU, METEOR, ROUGE-L, and SBERT, respectively.

Figures (5)

• Figure 1: Examples of code‑comment pair
• Figure 2: The explicit guidance for LLMs
• Figure 3: Overview of KUMIC
• Figure 4: The BLEU values of KUMIC with different hyperparameter values.
• Figure 5: The BLEU values of KUMIC with different quality scores.