"ENERGY STAR" LLM-Enabled Software Engineering Tools

TL;DR

The paper tackles the energy efficiency challenge of AI-enabled software engineering tools that incorporate LLM-based code generation in IDE-like environments. It introduces a Retrieval-Augmented Generation (RAG) framework combined with Prompt Engineering Techniques (PETs) to boost both code quality and energy efficiency, assessed across models from GPT-2 125M to CodeLlama 7B and Qwen 2.5 7B. Empirical results show that RAG can reduce energy consumption for some models (notably GPT-2 and CodeLlama) and even speed up inference in CodeLlama, while other models exhibit energy and latency trade-offs; importantly, smaller models can achieve competitive code quality when aided by RAG. The work provides practical guidance for deploying energy-aware LLM-assisted SE tools and sets the stage for broader evaluations with more powerful infrastructure and additional quality metrics.

Abstract

The discussion around AI-Engineering, that is, Software Engineering (SE) for AI-enabled Systems, cannot ignore a crucial class of software systems that are increasingly becoming AI-enhanced: Those used to enable or support the SE process, such as Computer-Aided SE (CASE) tools and Integrated Development Environments (IDEs). In this paper, we study the energy efficiency of these systems. As AI becomes seamlessly available in these tools and, in many cases, is active by default, we are entering a new era with significant implications for energy consumption patterns throughout the Software Development Lifecycle (SDLC). We focus on advanced Machine Learning (ML) capabilities provided by Large Language Models (LLMs). Our proposed approach combines Retrieval-Augmented Generation (RAG) with Prompt Engineering Techniques (PETs) to enhance both the quality and energy efficiency of LLM-based code generation. We present a comprehensive framework that measures real-time energy consumption and inference time across diverse model architectures ranging from 125M to 7B parameters, including GPT-2, CodeLlama, Qwen 2.5, and DeepSeek Coder. These LLMs, chosen for practical reasons, are sufficient to validate the core ideas and provide a proof of concept for more in-depth future analysis.