A Hybrid Swarm Intelligence Approach for Optimizing Multimodal Large Language Models Deployment in Edge-Cloud-based Federated Learning Environments

TL;DR

This work tackles the challenge of deploying multimodal large language models (MLLMs) in edge-cloud federated learning under resource constraints. It introduces a hybrid swarm intelligence framework that uses Particle Swarm Optimization (PSO) to select edge devices for MLLM deployment and Ant Colony Optimization (ACO) to optimize communication paths to the cloud, thereby reducing energy consumption, latency, and bandwidth usage. The approach demonstrates strong performance in simulations, achieving an accuracy of $92\%$, a $30\%$ reduction in communication cost, and higher client participation compared to traditional FL methods, while maintaining robustness to non-IID data. The results suggest that combining PSO and ACO with edge-cloud FL can enable scalable, energy-efficient deployment of MLLMs in real-world, privacy-preserving multimodal sensing environments, with future work extending to security, scalability, and broader application domains.

Abstract

The combination of Federated Learning (FL), Multimodal Large Language Models (MLLMs), and edge-cloud computing enables distributed and real-time data processing while preserving privacy across edge devices and cloud infrastructure. However, the deployment of MLLMs in FL environments with resource-constrained edge devices presents significant challenges, including resource management, communication overhead, and non-IID data. To address these challenges, we propose a novel hybrid framework wherein MLLMs are deployed on edge devices equipped with sufficient resources and battery life, while the majority of training occurs in the cloud. To identify suitable edge devices for deployment, we employ Particle Swarm Optimization (PSO), and Ant Colony Optimization (ACO) is utilized to optimize the transmission of model updates between edge and cloud nodes. This proposed swarm intelligence-based framework aims to enhance the efficiency of MLLM training by conducting extensive training in the cloud and fine-tuning at the edge, thereby reducing energy consumption and communication costs. Our experimental results show that the proposed method significantly improves system performance, achieving an accuracy of 92%, reducing communication cost by 30%, and enhancing client participation compared to traditional FL methods. These results make the proposed approach highly suitable for large-scale edge-cloud computing systems.

Figures (9)

• Figure 1: Edge-Cloud-based MLLM Architecture with Federated Learning for UAV Networks
• Figure 2: Architectural Overview of MLLM.
• Figure 3: Overview of the Proposed Framework for Optimizing MLLM Deployment Using Swarm Intelligence Techniques PSO and ACO
• Figure 4: The Architecture of the Use Case Solution
• Figure 5: Performance Metrics Comparison for Different Models