Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sustainability Is Not Linear: Quantifying Performance, Energy, and Privacy Trade-offs in On-Device Intelligence

Eziyo Ehsani, Luca Giamattei, Ivano Malavolta, Roberto Pietrantuono

Abstract

The migration of Large Language Models (LLMs) from cloud clusters to edge devices promises enhanced privacy and offline accessibility, but this transition encounters a harsh reality: the physical constraints of mobile batteries, thermal limits, and, most importantly, memory constraints. To navigate this landscape, we constructed a reproducible experimental pipeline to profile the complex interplay between energy consumption, latency, and quality. Unlike theoretical studies, we captured granular power metrics across eight models ranging from 0.5B to 9B parameters without requiring root access, ensuring our findings reflect realistic user conditions. We harness this pipeline to conduct an empirical case study on a flagship Android device, the Samsung Galaxy S25 Ultra, establishing foundational hypotheses regarding the trade-offs between generation quality, performance, and resource consumption. Our investigation uncovered a counter-intuitive quantization-energy paradox. While modern importance-aware quantization successfully reduces memory footprints to fit larger models into RAM, we found it yields negligible energy savings compared to standard mixed-precision methods. This proves that for battery life, the architecture of the model, not its quantization scheme, is the decisive factor. We further identified that Mixture-of-Experts (MoE) architectures defy the standard size-energy trend, offering the storage capacity of a 7B model while maintaining the lower energy profile of a 1B to 2B model. Finally, an analysis of these multi-objective trade-offs reveals a pragmatic sweet spot of mid-sized models, such as Qwen2.5-3B, that effectively balance response quality with sustainable energy consumption.

Sustainability Is Not Linear: Quantifying Performance, Energy, and Privacy Trade-offs in On-Device Intelligence

Abstract

The migration of Large Language Models (LLMs) from cloud clusters to edge devices promises enhanced privacy and offline accessibility, but this transition encounters a harsh reality: the physical constraints of mobile batteries, thermal limits, and, most importantly, memory constraints. To navigate this landscape, we constructed a reproducible experimental pipeline to profile the complex interplay between energy consumption, latency, and quality. Unlike theoretical studies, we captured granular power metrics across eight models ranging from 0.5B to 9B parameters without requiring root access, ensuring our findings reflect realistic user conditions. We harness this pipeline to conduct an empirical case study on a flagship Android device, the Samsung Galaxy S25 Ultra, establishing foundational hypotheses regarding the trade-offs between generation quality, performance, and resource consumption. Our investigation uncovered a counter-intuitive quantization-energy paradox. While modern importance-aware quantization successfully reduces memory footprints to fit larger models into RAM, we found it yields negligible energy savings compared to standard mixed-precision methods. This proves that for battery life, the architecture of the model, not its quantization scheme, is the decisive factor. We further identified that Mixture-of-Experts (MoE) architectures defy the standard size-energy trend, offering the storage capacity of a 7B model while maintaining the lower energy profile of a 1B to 2B model. Finally, an analysis of these multi-objective trade-offs reveals a pragmatic sweet spot of mid-sized models, such as Qwen2.5-3B, that effectively balance response quality with sustainable energy consumption.

Paper Structure

This paper contains 30 sections, 1 equation, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Energy Efficiency in NLP and Deployment Strategies
  4. Model Quantization for LLMs
  5. On-Device and Edge LLM Inference Systems
  6. Mobile Energy Measurement
  7. Automated Experimentation and Measurement
  8. Methodology
  9. Models and Quantization
  10. Experimental Setup and Orchestration
  11. Execution Pipeline and Reproducibility Controls
  12. Energy Measurement and Post-Processing
  13. Evaluation Metrics
  14. Results
  15. Performance Analysis
  16. ...and 15 more sections

Figures (9)

  • Figure 1: Q4_K_M: block-wise mixed precision with higher precision for more sensitive components.
  • Figure 2: IQ4_XS: importance-aware 4-bit quantization using calibration data and codebook-based reconstruction.
  • Figure 3: Controlled execution pipeline. Setup is executed once; measurement and inference are repeated per run with run-level isolation and thermal management.
  • Figure 4: Measurement workflow. A user-space monitoring app logs voltage, current, and temperature at fixed intervals while llama.cpp runs on-device. Logs are pulled to the host after each run.
  • Figure 5: Throughput across models and quantization schemes. Prefill benefits from parallel processing of the prompt, whereas generation is slower due to autoregressive decoding and higher per-token overhead.
  • ...and 4 more figures