Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Edge Large AI Models: Collaborative Deployment and IoT Applications

Zixin Wang, Yuanming Shi, Khaled. B. Letaief

TL;DR

Edge LAM deployment addresses latency and privacy by bringing large AI models to the network edge. The authors develop a collaborative training framework over heterogeneous edge networks (FedFT with LoRA and federated unlearning) and a microservice-based inference architecture that distributes LAM modules as edge microservices for MoE and CoT reasoning. They apply these approaches to multimodal IoT tasks, including intelligent transportation and industrial fault diagnosis, showing improvements in privacy, communication efficiency, and latency. The paper also discusses practical challenges and future directions in data sensing, energy efficiency, and generation quality.

Abstract

Large artificial intelligence models (LAMs) emulate human-like problem-solving capabilities across diverse domains, modalities, and tasks. By leveraging the communication and computation resources of geographically distributed edge devices, edge LAMs enable real-time intelligent services at the network edge. Unlike conventional edge AI, which relies on small or moderate-sized models for direct feature-to-prediction mappings, edge LAMs leverage the intricate coordination of modular components to enable context-aware generative tasks and multi-modal inference. We shall propose a collaborative deployment framework for edge LAM by characterizing the LAM intelligent capabilities and limited edge network resources. Specifically, we propose a collaborative training framework over heterogeneous edge networks that adaptively decomposes LAMs according to computation resources, data modalities, and training objectives, reducing communication and computation overheads during the fine-tuning process. Furthermore, we introduce a microservice-based inference framework that virtualizes the functional modules of edge LAMs according to their architectural characteristics, thereby improving resource utilization and reducing inference latency. The developed edge LAM will provide actionable solutions to enable diversified Internet-of-Things (IoT) applications, facilitated by constructing mappings from diverse sensor data to token representations and fine-tuning based on domain knowledge.

Edge Large AI Models: Collaborative Deployment and IoT Applications

TL;DR

Edge LAM deployment addresses latency and privacy by bringing large AI models to the network edge. The authors develop a collaborative training framework over heterogeneous edge networks (FedFT with LoRA and federated unlearning) and a microservice-based inference architecture that distributes LAM modules as edge microservices for MoE and CoT reasoning. They apply these approaches to multimodal IoT tasks, including intelligent transportation and industrial fault diagnosis, showing improvements in privacy, communication efficiency, and latency. The paper also discusses practical challenges and future directions in data sensing, energy efficiency, and generation quality.

Abstract

Large artificial intelligence models (LAMs) emulate human-like problem-solving capabilities across diverse domains, modalities, and tasks. By leveraging the communication and computation resources of geographically distributed edge devices, edge LAMs enable real-time intelligent services at the network edge. Unlike conventional edge AI, which relies on small or moderate-sized models for direct feature-to-prediction mappings, edge LAMs leverage the intricate coordination of modular components to enable context-aware generative tasks and multi-modal inference. We shall propose a collaborative deployment framework for edge LAM by characterizing the LAM intelligent capabilities and limited edge network resources. Specifically, we propose a collaborative training framework over heterogeneous edge networks that adaptively decomposes LAMs according to computation resources, data modalities, and training objectives, reducing communication and computation overheads during the fine-tuning process. Furthermore, we introduce a microservice-based inference framework that virtualizes the functional modules of edge LAMs according to their architectural characteristics, thereby improving resource utilization and reducing inference latency. The developed edge LAM will provide actionable solutions to enable diversified Internet-of-Things (IoT) applications, facilitated by constructing mappings from diverse sensor data to token representations and fine-tuning based on domain knowledge.
Paper Structure (13 sections, 5 figures)

This paper contains 13 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Collaborative training for Edge LAM over Heterogeneous Edge Networks
  3. Heterogeneous Federated Fine-tuning
  4. Federated Unlearning for Edge LAM
  5. Microservice enabled Edge LAM Inference
  6. Microservice for Mixture-of-Experts Edge LAM Inference
  7. Microservice for Chain-of-Thought Edge LAM Inference
  8. Case Study
  9. Edge LAM For IoT Applications
  10. Multimodal Edge LAM for Intelligent Transportation
  11. Federated Edge LAM for Trustworthy Industrial IoT
  12. Challenges and Future Directions
  13. Conclusions

Figures (5)

  • Figure 1: Summary: challenges, techniques, and advantages.
  • Figure 2: Federated fine-tuning over heterogeneous wireless networks.
  • Figure 3: Wireless federated unlearning for edge LAMs.
  • Figure 4: Microservice architecture for edge LAM inference with MoE.
  • Figure 5: Performance comparison in reasoning with CoT.