Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative AI as a Service in 6G Edge-Cloud: Generation Task Offloading by In-context Learning

Hao Zhou, Chengming Hu, Dun Yuan, Ye Yuan, Di Wu, Xue Liu, Zhu Han, Charlie Zhang

TL;DR

This paper tackles end-to-end delay in GAI services over 6G networks by proposing an edge-cloud deployment that assigns generation tasks to edge LLMs or central-cloud LLMs. It formalizes a joint radio-resource and offloading optimization, and introduces an in-context learning framework to make offloading decisions without additional model training. The approach demonstrates that edge-cloud collaboration with ICL can achieve generation service quality comparable to DRL baselines, while reducing the need for dedicated training. It also acknowledges potential security vulnerabilities and suggests future work on robust deployment strategies for reliable 6G GAI-enabled services.

Abstract

Generative artificial intelligence (GAI) is a promising technique towards 6G networks, and generative foundation models such as large language models (LLMs) have attracted considerable interest from academia and telecom industry. This work considers a novel edge-cloud deployment of foundation models in 6G networks. Specifically, it aims to minimize the service delay of foundation models by radio resource allocation and task offloading, i.e., offloading diverse content generation tasks to proper LLMs at the network edge or cloud. In particular, we first introduce the communication system model, i.e., allocating radio resources and calculating link capacity to support generated content transmission, and then we present the LLM inference model to calculate the delay of content generation. After that, we propose a novel in-context learning method to optimize the task offloading decisions. It utilizes LLM's inference capabilities, and avoids the difficulty of dedicated model training or fine-tuning as in conventional machine learning algorithms. Finally, the simulations demonstrate that the proposed edge-cloud deployment and in-context learning task offloading method can achieve satisfactory generation service quality without dedicated model training or fine-tuning.

Generative AI as a Service in 6G Edge-Cloud: Generation Task Offloading by In-context Learning

TL;DR

This paper tackles end-to-end delay in GAI services over 6G networks by proposing an edge-cloud deployment that assigns generation tasks to edge LLMs or central-cloud LLMs. It formalizes a joint radio-resource and offloading optimization, and introduces an in-context learning framework to make offloading decisions without additional model training. The approach demonstrates that edge-cloud collaboration with ICL can achieve generation service quality comparable to DRL baselines, while reducing the need for dedicated training. It also acknowledges potential security vulnerabilities and suggests future work on robust deployment strategies for reliable 6G GAI-enabled services.

Abstract

Generative artificial intelligence (GAI) is a promising technique towards 6G networks, and generative foundation models such as large language models (LLMs) have attracted considerable interest from academia and telecom industry. This work considers a novel edge-cloud deployment of foundation models in 6G networks. Specifically, it aims to minimize the service delay of foundation models by radio resource allocation and task offloading, i.e., offloading diverse content generation tasks to proper LLMs at the network edge or cloud. In particular, we first introduce the communication system model, i.e., allocating radio resources and calculating link capacity to support generated content transmission, and then we present the LLM inference model to calculate the delay of content generation. After that, we propose a novel in-context learning method to optimize the task offloading decisions. It utilizes LLM's inference capabilities, and avoids the difficulty of dedicated model training or fine-tuning as in conventional machine learning algorithms. Finally, the simulations demonstrate that the proposed edge-cloud deployment and in-context learning task offloading method can achieve satisfactory generation service quality without dedicated model training or fine-tuning.
Paper Structure (12 sections, 10 equations, 5 figures)

This paper contains 12 sections, 10 equations, 5 figures.

Table of Contents

  1. Introduction
  2. System Model
  3. Communication Model for Content Transmission
  4. LLM Inference Model for Content Generation
  5. Problem Formulation
  6. In-context Learning for Task Offloading
  7. Prompting System Design
  8. Prioritized Experience Replay and Exploration Strategies
  9. Performance Evaluation
  10. Simulation Settings
  11. Simulation Results
  12. Conclusion

Figures (5)

  • Figure 1: GAI model deployment and services in wireless networks.
  • Figure 2: LLM inference process illustration. (EOS: end-of-sequence).
  • Figure 3: LLM-enabled in-context learning for task offloading.
  • Figure 4: The overall optimization procedure of the proposed technique
  • Figure 5: Simulation results and comparisons