Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating the Environmental Impact of using SLMs and Prompt Engineering for Code Generation

Md Afif Al Mamun, Sayan Nath, Gias Uddin, Novarun Deb

Abstract

The shift from cloud-hosted Large Language Models (LLMs) to locally deployed open-source Small Language Models (SLMs) has democratized AI-assisted coding; however, it has also decentralized the environmental footprint of AI. While prompting strategies - such as Chain-of-Thought and ReAct - serve as external mechanisms for optimizing code generation without modifying model parameters, their impact on energy consumption and carbon emissions remains largely invisible to developers. This paper presents the first systematic empirical study investigating how different prompt engineering strategies in SLM-based code generation impact code generation accuracy alongside sustainability factors. We evaluate six prominent prompting strategies across 11 open-source models (ranging from 1B to 34B parameters) using the HumanEval+ and MBPP+ benchmarks. By measuring Pass@1 accuracy alongside energy (kWh), carbon emissions (kgCO2eq), and inference latency, we reveal that sustainability often decouples from accuracy, allowing significant environmental optimizations without sacrificing performance. Our findings indicate that Chain-of-Thought, being a simpler prompting technique, can provide a near-optimal balance between reasoning capability and energy efficiency. Conversely, multi-sampling strategies often incur disproportionate costs for marginal gains. Finally, we identify grid carbon intensity as the dominant factor in deployment-time emissions, highlighting the need for practitioners to consider regional energy profiles. This work provides a quantitative foundation for "green" prompt engineering, enabling developers to align high-performance code generation with ecological responsibility.

Evaluating the Environmental Impact of using SLMs and Prompt Engineering for Code Generation

Abstract

The shift from cloud-hosted Large Language Models (LLMs) to locally deployed open-source Small Language Models (SLMs) has democratized AI-assisted coding; however, it has also decentralized the environmental footprint of AI. While prompting strategies - such as Chain-of-Thought and ReAct - serve as external mechanisms for optimizing code generation without modifying model parameters, their impact on energy consumption and carbon emissions remains largely invisible to developers. This paper presents the first systematic empirical study investigating how different prompt engineering strategies in SLM-based code generation impact code generation accuracy alongside sustainability factors. We evaluate six prominent prompting strategies across 11 open-source models (ranging from 1B to 34B parameters) using the HumanEval+ and MBPP+ benchmarks. By measuring Pass@1 accuracy alongside energy (kWh), carbon emissions (kgCO2eq), and inference latency, we reveal that sustainability often decouples from accuracy, allowing significant environmental optimizations without sacrificing performance. Our findings indicate that Chain-of-Thought, being a simpler prompting technique, can provide a near-optimal balance between reasoning capability and energy efficiency. Conversely, multi-sampling strategies often incur disproportionate costs for marginal gains. Finally, we identify grid carbon intensity as the dominant factor in deployment-time emissions, highlighting the need for practitioners to consider regional energy profiles. This work provides a quantitative foundation for "green" prompt engineering, enabling developers to align high-performance code generation with ecological responsibility.

Paper Structure

This paper contains 24 sections, 6 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Small Language Models for Code Generation
  4. Prompt Engineering
  5. Energy Efficiency and Environmental Impact of Code Generation Models.
  6. Research Methodology
  7. Research Question (RQ)
  8. Dataset
  9. Small Language Models
  10. Prompting Strategies
  11. Evaluation Framework Setup
  12. Carbon Emission Estimation (CO$_2$eq)
  13. Evaluation Metrics
  14. Results
  15. RQ1: Accuracy-Emissions Relationship Across SLMs
  16. ...and 9 more sections

Figures (7)

  • Figure 1: Evaluation framework to benchmark LLMs.
  • Figure 2: Comparison of Pass@1 accuracy and CO$_2$ emissions on MBPP+ and HumanEval+ across models.
  • Figure 3: Bubble chart of mean Pass@1 accuracy versus energy consumption for different prompting strategies. Bubble size represents average token count and color represents average CO$_2$ emissions.
  • Figure 4: Comparison of CO$_2$ emission, energy consumption, and inference time of different models in Machine 1 and Machine 2 when Chain-of-Thought is used.
  • Figure 5: Relationship between different sustainability factors across different prompting strategies.
  • ...and 2 more figures