JPPO: Joint Power and Prompt Optimization for Accelerated Large Language Model Services

TL;DR

This work tackles the challenge of deploying large language model services over wireless networks where long prompts impose heavy communication and computation costs. It introduces JPPO, a framework that fuses edge SLM-based prompt compression with joint wireless power optimization, solved via a centralized Double DQN to adapt compression and transmit power to channel conditions and prompt content. Key contributions include a formal prompt structure and fidelity-based QoS metrics, an energy-delay-aware system model, and a DRL solution that balances representation fidelity, transmission reliability, and energy usage. Empirical results show notable latency reductions (about 17%), high prompt fidelity (≈0.9), low BER (<0.2), and stable power consumption (4–5 W), demonstrating the practicality of efficient wireless LLM services. Overall, JPPO provides a principled approach to edge-assisted LLM inference that can inform real-time deployments in IoT and edge computing ecosystems.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, leading to their increasing deployment in wireless networks for a wide variety of user services. However, the growing longer prompt setting highlights the crucial issue of computational resource demands and huge communication load. To address this challenge, we propose Joint Power and Prompt Optimization (JPPO), a framework that combines Small Language Model (SLM)-based prompt compression with wireless power allocation optimization. By deploying SLM at user devices for prompt compression and employing Deep Reinforcement Learning for joint optimization of compression ratio and transmission power, JPPO effectively balances service quality with resource efficiency. Experimental results demonstrate that our framework achieves high service fidelity and low bit error rates while optimizing power usage in wireless LLM services. The system reduces response time by about 17%, with the improvement varying based on the length of the original prompt.

Figures (5)

• Figure 1: Time consumption comparison of first token generation between conventional network-aided LLM inference service architecture and our design under long user prompts.
• Figure 2: System model of wireless network-aided LLM services and overview of our proposed JPPO, where user-generated long prompts are first compressed through SLM-based edge computing, then transmitted with optimized power allocation via wireless networks to LLM server, and finally inference results are returned to users.
• Figure 3: The example illustrates wireless network-aided LLM services with SLM-based prompt compression, with a 4x compression ratio. The highlighted parts represent key information from the original prompt. Additionally, we show the corresponding steps of measuring the three sub-performance metrics ($\mathbf{f_1}$, $\mathbf{f_2}$ and $\mathbf{f_3}$) of the fidelity metric $\mathbf{f}$ throughout the process.
• Figure 4: The convergence performance of reward for the proposed Double DQN algorithm.
• Figure 5: The training performance of the proposed Double DQN algorithm. Figs. 5(a), 5(b), and 5(c) show the fidelity performance, the BER, and the power consumption, respectively.