Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are Human Rules Necessary? Generating Reusable APIs with CoT Reasoning and In-Context Learning

Yubo Mai, Zhipeng Gao, Xing Hu, Lingfeng Bao, Yu Liu, Jianling Sun

TL;DR

Code2API tackles the APIzation problem for Stack Overflow snippets by replacing labor-intensive rule-based methods with a prompt-driven approach that uses chain-of-thought reasoning and few-shot learning to elicit developer-like API construction from LLMs. It outperforms the prior state of the art APIzator in parameter and return inference, yields higher-quality, more expressive method names, and generalizes well to Python with strong compilation rates. The work provides a practical Chrome extension and introduces large-scale Java and Python API datasets, underscoring the approach's practicality for rapid API reuse. Overall, Code2API demonstrates that careful prompt design can achieve human-competitive reusable API generation across languages, with meaningful implications for software reuse and developer tooling.

Abstract

Inspired by the great potential of Large Language Models (LLMs) for solving complex coding tasks, in this paper, we propose a novel approach, named Code2API, to automatically perform APIzation for Stack Overflow code snippets. Code2API does not require additional model training or any manual crafting rules and can be easily deployed on personal computers without relying on other external tools. Specifically, Code2API guides the LLMs through well-designed prompts to generate well-formed APIs for given code snippets. To elicit knowledge and logical reasoning from LLMs, we used chain-of-thought (CoT) reasoning and few-shot in-context learning, which can help the LLMs fully understand the APIzation task and solve it step by step in a manner similar to a developer. Our evaluations show that Code2API achieves a remarkable accuracy in identifying method parameters (65%) and return statements (66%) equivalent to human-generated ones, surpassing the current state-of-the-art approach, APIzator, by 15.0% and 16.5% respectively. Moreover, compared with APIzator, our user study demonstrates that Code2API exhibits superior performance in generating meaningful method names, even surpassing the human-level performance, and developers are more willing to use APIs generated by our approach, highlighting the applicability of our tool in practice. Finally, we successfully extend our framework to the Python dataset, achieving a comparable performance with Java, which verifies the generalizability of our tool.

Are Human Rules Necessary? Generating Reusable APIs with CoT Reasoning and In-Context Learning

TL;DR

Code2API tackles the APIzation problem for Stack Overflow snippets by replacing labor-intensive rule-based methods with a prompt-driven approach that uses chain-of-thought reasoning and few-shot learning to elicit developer-like API construction from LLMs. It outperforms the prior state of the art APIzator in parameter and return inference, yields higher-quality, more expressive method names, and generalizes well to Python with strong compilation rates. The work provides a practical Chrome extension and introduces large-scale Java and Python API datasets, underscoring the approach's practicality for rapid API reuse. Overall, Code2API demonstrates that careful prompt design can achieve human-competitive reusable API generation across languages, with meaningful implications for software reuse and developer tooling.

Abstract

Inspired by the great potential of Large Language Models (LLMs) for solving complex coding tasks, in this paper, we propose a novel approach, named Code2API, to automatically perform APIzation for Stack Overflow code snippets. Code2API does not require additional model training or any manual crafting rules and can be easily deployed on personal computers without relying on other external tools. Specifically, Code2API guides the LLMs through well-designed prompts to generate well-formed APIs for given code snippets. To elicit knowledge and logical reasoning from LLMs, we used chain-of-thought (CoT) reasoning and few-shot in-context learning, which can help the LLMs fully understand the APIzation task and solve it step by step in a manner similar to a developer. Our evaluations show that Code2API achieves a remarkable accuracy in identifying method parameters (65%) and return statements (66%) equivalent to human-generated ones, surpassing the current state-of-the-art approach, APIzator, by 15.0% and 16.5% respectively. Moreover, compared with APIzator, our user study demonstrates that Code2API exhibits superior performance in generating meaningful method names, even surpassing the human-level performance, and developers are more willing to use APIs generated by our approach, highlighting the applicability of our tool in practice. Finally, we successfully extend our framework to the Python dataset, achieving a comparable performance with Java, which verifies the generalizability of our tool.
Paper Structure (39 sections, 7 figures, 6 tables)

This paper contains 39 sections, 7 figures, 6 tables.

Table of Contents

  1. INTRODUCTION
  2. Preliminary Study
  3. Limitations of APIzator
  4. Motivating Examples
  5. OUR APPROACH
  6. Data Preparing
  7. Prompt Engineering
  8. Prompt Role Designation.
  9. Chain-of-Thought Reasoning.
  10. Few-Shot Learning.
  11. Prompt construction.
  12. The Implementation of the LLM
  13. Empirical Evaluation
  14. RQ-1: Method Parameters and Return Statements Evaluation
  15. Experimental Setup.
  16. ...and 24 more sections

Figures (7)

  • Figure 1: Human APIzation of A SO Code Snippet.
  • Figure 2: API Examples Generated by APIzator.
  • Figure 3: The Workflow of Our Approach.
  • Figure 4: An Example of Reasonable APIs generated by Code2API.
  • Figure 5: An Example of Better APIs Generated by Code2API.
  • ...and 2 more figures