Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Code Runtime Performance through Context-Aware Retrieval-Augmented Generation

Manish Acharya, Yifan Zhang, Kevin Leach, Yu Huang

TL;DR

Automating code runtime optimization with large language models remains challenging due to insufficient deep program analysis. AutoPatch combines human-inspired insights, CFG-diff analysis, and retrieval-augmented prompting to produce context-aware patches, formalizing CFG differences as $\Delta G = G_p - G_o$ with components $\Delta S$, $\Delta F$, and $\Delta C$. In experiments on IBM Project CodeNet (C++), AutoPatch achieves a 7.3% runtime improvement over GPT-4o and demonstrates lexical improvements across multiple metrics, validating a unified pipeline that bridges human expertise and automated optimization. This framework lays the groundwork for scalable, interpretable, and safer automated program optimization across diverse software engineering tasks.

Abstract

Optimizing software performance through automated code refinement offers a promising avenue for enhancing execution speed and efficiency. Despite recent advancements in LLMs, a significant gap remains in their ability to perform in-depth program analysis. This study introduces AUTOPATCH, an in-context learning approach designed to bridge this gap by enabling LLMs to automatically generate optimized code. Inspired by how programmers learn and apply knowledge to optimize software, AUTOPATCH incorporates three key components: (1) an analogy-driven framework to align LLM optimization with human cognitive processes, (2) a unified approach that integrates historical code examples and CFG analysis for context-aware learning, and (3) an automated pipeline for generating optimized code through in-context prompting. Experimental results demonstrate that AUTOPATCH achieves a 7.3% improvement in execution efficiency over GPT-4o across common generated executable code, highlighting its potential to advance automated program runtime optimization.

Optimizing Code Runtime Performance through Context-Aware Retrieval-Augmented Generation

TL;DR

Automating code runtime optimization with large language models remains challenging due to insufficient deep program analysis. AutoPatch combines human-inspired insights, CFG-diff analysis, and retrieval-augmented prompting to produce context-aware patches, formalizing CFG differences as with components , , and . In experiments on IBM Project CodeNet (C++), AutoPatch achieves a 7.3% runtime improvement over GPT-4o and demonstrates lexical improvements across multiple metrics, validating a unified pipeline that bridges human expertise and automated optimization. This framework lays the groundwork for scalable, interpretable, and safer automated program optimization across diverse software engineering tasks.

Abstract

Optimizing software performance through automated code refinement offers a promising avenue for enhancing execution speed and efficiency. Despite recent advancements in LLMs, a significant gap remains in their ability to perform in-depth program analysis. This study introduces AUTOPATCH, an in-context learning approach designed to bridge this gap by enabling LLMs to automatically generate optimized code. Inspired by how programmers learn and apply knowledge to optimize software, AUTOPATCH incorporates three key components: (1) an analogy-driven framework to align LLM optimization with human cognitive processes, (2) a unified approach that integrates historical code examples and CFG analysis for context-aware learning, and (3) an automated pipeline for generating optimized code through in-context prompting. Experimental results demonstrate that AUTOPATCH achieves a 7.3% improvement in execution efficiency over GPT-4o across common generated executable code, highlighting its potential to advance automated program runtime optimization.

Paper Structure

This paper contains 26 sections, 2 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Studying Human Behavior in Code Optimization
  4. Understanding Control Flow through Programmer Analysis
  5. Extracting Optimization Patterns from Insights
  6. Generalizing Knowledge for Actionable Strategies
  7. Leveraging Historical Insights for LLM Optimization
  8. CFG Diff as Analytical Basis
  9. Prompting the LLM with CFG Diff
  10. Retrieval-Augmented Guidance
  11. Combining Retrieved Insights for Code Optimization
  12. Experimental Design
  13. Experiment Preparation
  14. Baseline Methods
  15. Evaluation Metrics
  16. ...and 11 more sections

Figures (2)

  • Figure 1: Overview of the AutoPatch Workflow: AutoPatch mimics human cognitive processes to optimize code using historical examples and CFG analysis. ① represents the programmer learning phase, extracting insights from historical code and CFG differences. ② integrates CFG analysis and optimization patterns into a context-aware LLM. ③ applies this knowledge to generate optimized code for new inputs.
  • Figure 2: Average execution times for different optimization types across context-based, naive, and zero-shot generation methods. Context-based generation generally achieves lower execution times, except for a slight underperformance in performance enhancement.