ECO: Enhanced Code Optimization via Performance-Aware Prompting for Code-LLMs

TL;DR

ECO addresses code runtime optimization by shifting from relying solely on slow-fast code pairs to performance-aware prompting. It distills runtime optimization instructions (ROIs) from reference pairs and uses a Symbolic Advisor plus ROI Retriever to produce bottleneck-aware prompts for code-LLMs without fine-tuning. Across PIE and Codeforces benchmarks, ECO yields substantial speedups (up to $7.81\times$ in some cases) with minimal correctness loss, and generalizes across model scales and closed/open models. The framework is model-agnostic and plug-in, enabling practical deployment, and is supported by reproducibility resources including prompts and ROI data.

Abstract

Code runtime optimization-the task of rewriting a given code to a faster one-remains challenging, as it requires reasoning about performance trade-offs involving algorithmic and structural choices. Recent approaches employ code-LLMs with slow-fast code pairs provided as optimization guidance, but such pair-based methods obscure the causal factors of performance gains and often lead to superficial pattern imitation rather than genuine performance reasoning. We introduce ECO, a performance-aware prompting framework for code optimization. ECO first distills runtime optimization instructions (ROIs) from reference slow-fast code pairs; Each ROI describes root causes of inefficiency and the rationales that drive performance improvements. For a given input code, ECO in parallel employs (i) a symbolic advisor to produce a bottleneck diagnosis tailored to the code, and (ii) an ROI retriever to return related ROIs. These two outputs are then composed into a performance-aware prompt, providing actionable guidance for code-LLMs. ECO's prompts are model-agnostic, require no fine-tuning, and can be easily prepended to any code-LLM prompt. Our empirical studies highlight that ECO prompting significantly improves code-LLMs' ability to generate efficient code, achieving speedups of up to 7.81x while minimizing correctness loss.

Figures (14)

• Figure 1: Comparison of ECO’s performance-aware prompt with conventional retrieval.
• Figure 2: Overview of our ECO framework. ECO locates bottleneck code snippets using the symbolic advisor and the strategy retriever module, and then generates directives for code-LLMs.
• Figure 3: Detailed analysis of errors ($\downarrow$) and correct but not optimized (-), and optimized ($\uparrow$) cases for each method. Lower error percentages and higher optimized percentages are better.
• Figure 4: (Left) Distribution of bottlenecks detected by the symbolic advisor. (Right) TF-IDF ranking of the top-10 overlapping keywords between input and retrieved code, under RAG and ROI retriever on the PIE test set.
• Figure 5: Example of a runtime optimization instruction.