APIGen: Generative API Method Recommendation
Yujia Chen, Cuiyun Gao, Muyijie Zhu, Qing Liao, Yong Wang, Guoai Xu
TL;DR
APIGen tackles the challenge of suggesting appropriate APIs for programming queries by leveraging enhanced in-context learning. It introduces a novel three-part pipeline—diverse example retrieval, intent-aware prompt construction, and guided API recommendation—that enables large language models to reason about task intent and API knowledge before generating recommendations. Through extensive experiments on APIBENCH-Q and BIKER-Dataset, APIGen outperforms the state-of-the-art baselines across method- and class-level metrics, with ablations showing both example retrieval and reasoning prompts are crucial. The work demonstrates strong cross-model generalizability, improving performance for multiple LLMs including GPT-3.5, ChatGPT, and GPT-4, and provides open-source code for reproducibility and further research in code intelligence. The practical impact lies in delivering more accurate, interpretable API recommendations with fewer labeled data requirements, aided by rich demonstrations and reasoning guided by structured intent and API knowledge.
Abstract
Automatic API method recommendation is an essential task of code intelligence, which aims to suggest suitable APIs for programming queries. Existing approaches can be categorized into two primary groups: retrieval-based and learning-based approaches. Although these approaches have achieved remarkable success, they still come with notable limitations. The retrieval-based approaches rely on the text representation capabilities of embedding models, while the learning-based approaches require extensive task-specific labeled data for training. To mitigate the limitations, we propose APIGen, a generative API recommendation approach through enhanced in-context learning (ICL). APIGen involves two main components: (1) Diverse Examples Selection. APIGen searches for similar posts to the programming queries from the lexical, syntactical, and semantic perspectives, providing more informative examples for ICL. (2) Guided API Recommendation. APIGen enables large language models (LLMs) to perform reasoning before generating API recommendations, where the reasoning involves fine-grained matching between the task intent behind the queries and the factual knowledge of the APIs. With the reasoning process, APIGen makes recommended APIs better meet the programming requirement of queries and also enhances the interpretability of results. We compare APIGen with four existing approaches on two publicly available benchmarks. Experiments show that APIGen outperforms the best baseline CLEAR by 105.8% in method-level API recommendation and 54.3% in class-level API recommendation in terms of SuccessRate@1. Besides, APIGen achieves an average 49.87% increase compared to the zero-shot performance of popular LLMs such as GPT-4 in method-level API recommendation regarding the SuccessRate@3 metric.