Large Language Models (LLMs): Deployment, Tokenomics and Sustainability

TL;DR

This paper surveys the deployment, tokenomics, and sustainability of state-of-the-art LLMs, comparing Retrieval-Augmented Generation (RAG) and fine-tuning for domain adaptation and discussing their trade-offs. It quantifies hardware needs for training and inference across xPU architectures, introduces QoE-based metrics for user-perceived performance, and models latency as $Latency = TTFT + TOPT \cdot N$ to illuminate cost-performance trade-offs. The authors envision a hybrid edge-centric LLM architecture that brings inference closer to users while maintaining accuracy via cloud validation, and they assess environmental and economic sustainability using tools like LLMCarbon and mlco2. The work highlights practical considerations for responsible deployment, balancing latency, cost, performance, and carbon footprint in real-world settings.

Abstract

The rapid advancement of Large Language Models (LLMs) has significantly impacted human-computer interaction, epitomized by the release of GPT-4o, which introduced comprehensive multi-modality capabilities. In this paper, we first explored the deployment strategies, economic considerations, and sustainability challenges associated with the state-of-the-art LLMs. More specifically, we discussed the deployment debate between Retrieval-Augmented Generation (RAG) and fine-tuning, highlighting their respective advantages and limitations. After that, we quantitatively analyzed the requirement of xPUs in training and inference. Additionally, for the tokenomics of LLM services, we examined the balance between performance and cost from the quality of experience (QoE)'s perspective of end users. Lastly, we envisioned the future hybrid architecture of LLM processing and its corresponding sustainability concerns, particularly in the environmental carbon footprint impact. Through these discussions, we provided a comprehensive overview of the operational and strategic considerations essential for the responsible development and deployment of LLMs.

Figures (4)

• Figure 1: LLMs will be the computational core to interact with multimodal multimedia, legacy file system, components of PC, and other LLMs.
• Figure 2: Retrieval-Augmented Generation (RAG) architecture for large language models. RAG retrieves relevant data from a knowledge base and optionally few-shot examples to address context window limitations and improve response generation.
• Figure 3: LLM fine-tuning for a specific task. The pre-trained model has learned parameters from a massive dataset, while the parameters are updated through fine-tuning in a smaller target dataset. The red dots are the learned parameters in the original models, while the green dots are the new parameters learned from LoRA training. RLHF aligns the model for optimal performance.
• Figure 4: The hybrid architecture of AI processing of LLMs. The central and edge clouds and devices work together to deliver high QoE LLM service by balancing factors, including inference accuracy, latency, device capacity, privacy, and security.