A Review on Edge Large Language Models: Design, Execution, and Applications

TL;DR

This survey addresses the challenge of running large language models on resource-constrained edge devices by presenting a complete optimization pipeline that spans offline pre-deployment techniques (quantization, pruning, distillation, low-rank, and complementary methods), online runtime optimizations (software-level, hardware-software co-design, and hardware-level), and practical on-device applications across personal, enterprise, and industrial domains. It emphasizes how careful model compression combined with adaptive runtime strategies can bridge the gap between the rapid growth of LLM capabilities and the modest performance of edge hardware, enabling fast, private, and robust on-device inference. Key contributions include a structured taxonomy, representative techniques and systems, and guidance on selecting and combining methods to meet diverse latency, memory, and energy constraints. The work underscores the practical impact by detailing real-world applications and outlining future directions that address heterogeneity, fault tolerance, and continual learning for scalable edge AI.

Abstract

Large language models (LLMs) have revolutionized natural language processing with their exceptional understanding, synthesizing, and reasoning capabilities. However, deploying LLMs on resource-constrained edge devices presents significant challenges due to computational limitations, memory constraints, and edge hardware heterogeneity. This survey provides a comprehensive overview of recent advancements in edge LLMs, covering the entire lifecycle: from resource-efficient model design and pre-deployment strategies to runtime inference optimizations. It also explores on-device applications across various domains. By synthesizing state-of-the-art techniques and identifying future research directions, this survey bridges the gap between the immense potential of LLMs and the constraints of edge computing.

Figures (11)

• Figure 1: Edge LLMs deployment, optimization and application pipeline.
• Figure 2: A comprehensive overview of on-device LLMs, with detailed research categorized into: Pre-Deployment Techniques (§ \ref{['Sec: Offline Pre-Deployment Model Design Techniques']}, research details in Fig. \ref{['fig: fig_sec3_offline_overview']}), Runtime Optimizations (§ \ref{['Sec: On-Device LLM-Based Applications']}, details distributed across Tables \ref{['tab: tab_collaboration_literature']}-\ref{['tab: tab_hardware']}), and On-Device Applications (§ \ref{['Sec: On-Device LLM-Based Applications']}, research details in Fig. \ref{['fig: fig_sec5_application']}). Each branch represents a key research direction with detailed methodology and implementation discussions in corresponding sections.
• Figure 3: LLMs (TFLOPs) vs Edge Devices (TOPS) Over Time. (QSD: Qualcomm Snapdragon.)
• Figure 4: Temporal Distribution of On-Device LLM Research.
• Figure 5: An Overview of Offline Pre-deployment Model Design Techniques and Literature.