Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization

Dong Huang, Jianbo Dai, Han Weng, Puzhen Wu, Yuhao Qing, Heming Cui, Zhijiang Guo, Jie M. Zhang

TL;DR

Effi-Learner introduces a self-optimization loop that uses locally gathered execution overhead profiles to iteratively refine LLM-generated code for improved runtime and memory efficiency. By profiling per-line execution time and memory usage, the system guides the LLM to apply targeted optimizations while preserving correctness, demonstrably reducing execution time and memory footprints across diverse models and benchmarks. The approach is model-agnostic and shows strong generalizability across datasets (EffiBench, HumanEval, MBPP) and both open-source and closed-source LLMs, with source code released for reproducibility. The work highlights the practical value of overhead-aware feedback in transforming code quality beyond functional correctness, while also acknowledging trade-offs such as increased profiling cost and tokens consumed during optimization. Overall, Effi-Learner offers a scalable path to deploying more resource-efficient AI-assisted code generation in real-world settings.

Abstract

Large language models (LLMs) have shown remarkable progress in code generation, but their generated code often suffers from inefficiency, resulting in longer execution times and higher memory consumption. To address this issue, we propose \textbf{EffiLearner}, a self-optimization framework that utilizes execution overhead profiles to improve the efficiency of LLM-generated code. EffiLearner first generates code using an LLM, then executes it locally to capture execution time and memory usage profiles. These profiles are fed back to the LLM, which then revises the code to reduce overhead. To evaluate the effectiveness of EffiLearner, we conduct extensive experiments on the EffiBench, HumanEval, and MBPP with 16 open-source and 6 closed-source models. Our evaluation results demonstrate that through iterative self-optimization, EffiLearner significantly enhances the efficiency of LLM-generated code. For example, the execution time (ET) of StarCoder2-15B for the EffiBench decreases from 0.93 (s) to 0.12 (s) which reduces 87.1% the execution time requirement compared with the initial code. The total memory usage (TMU) of StarCoder2-15B also decreases from 22.02 (Mb*s) to 2.03 (Mb*s), which decreases 90.8% of total memory consumption during the execution process. The source code of EffiLearner was released in https://github.com/huangd1999/EffiLearner

EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization

TL;DR

Effi-Learner introduces a self-optimization loop that uses locally gathered execution overhead profiles to iteratively refine LLM-generated code for improved runtime and memory efficiency. By profiling per-line execution time and memory usage, the system guides the LLM to apply targeted optimizations while preserving correctness, demonstrably reducing execution time and memory footprints across diverse models and benchmarks. The approach is model-agnostic and shows strong generalizability across datasets (EffiBench, HumanEval, MBPP) and both open-source and closed-source LLMs, with source code released for reproducibility. The work highlights the practical value of overhead-aware feedback in transforming code quality beyond functional correctness, while also acknowledging trade-offs such as increased profiling cost and tokens consumed during optimization. Overall, Effi-Learner offers a scalable path to deploying more resource-efficient AI-assisted code generation in real-world settings.

Abstract

Large language models (LLMs) have shown remarkable progress in code generation, but their generated code often suffers from inefficiency, resulting in longer execution times and higher memory consumption. To address this issue, we propose \textbf{EffiLearner}, a self-optimization framework that utilizes execution overhead profiles to improve the efficiency of LLM-generated code. EffiLearner first generates code using an LLM, then executes it locally to capture execution time and memory usage profiles. These profiles are fed back to the LLM, which then revises the code to reduce overhead. To evaluate the effectiveness of EffiLearner, we conduct extensive experiments on the EffiBench, HumanEval, and MBPP with 16 open-source and 6 closed-source models. Our evaluation results demonstrate that through iterative self-optimization, EffiLearner significantly enhances the efficiency of LLM-generated code. For example, the execution time (ET) of StarCoder2-15B for the EffiBench decreases from 0.93 (s) to 0.12 (s) which reduces 87.1% the execution time requirement compared with the initial code. The total memory usage (TMU) of StarCoder2-15B also decreases from 22.02 (Mb*s) to 2.03 (Mb*s), which decreases 90.8% of total memory consumption during the execution process. The source code of EffiLearner was released in https://github.com/huangd1999/EffiLearner
Paper Structure (34 sections, 6 equations, 18 figures, 9 tables)

This paper contains 34 sections, 6 equations, 18 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs for Code Generation
  4. Learning From Feedback
  5. Effi-Learner
  6. Code Generation
  7. Overhead Profiling
  8. Code Refinement
  9. Prompt Construction
  10. Evaluation
  11. Dataset and Metrics
  12. Implementation Details
  13. Main Results
  14. Impact of Self-Optimization Steps
  15. Feedback of Overhead Profile
  16. ...and 19 more sections

Figures (18)

  • Figure 1: A case for the task with code and Effi-Learner refined version. The lower left panel shows the initial completion generated by an LLM, its profile shows its inefficiency. The lower right panel shows the final efficient answer output by applying Effi-Learner.
  • Figure 2: Pipeline of Effi-Learner. LLMs first generate code for the given problem. This code is then executed locally to gather overhead profiles. These profiles are subsequently utilized by the LLMs to optimize the code in successive iterations, thereby enhancing the overall efficiency of the generated code. A comprehensive illustration is provided in the Appendix \ref{['fig:effective_case_task_description']}-\ref{['fig:effective_case_task_refine_code_memory']}.
  • Figure 3: Prompt template used by Effi-Learner in the self-optimization stage.
  • Figure 4: A case illustration of GPT-4-turbo-preview (OpenLock). Part A: Task Description.
  • Figure 5: A case illustration of GPT-4-turbo-preview (OpenLock). Part B: GPT-4-turbo-preview generated code.
  • ...and 13 more figures