LLM-Driven Intent-Based Privacy-Aware Orchestration Across the Cloud-Edge Continuum

TL;DR

The paper addresses privacy-preserving orchestration across the cloud-edge continuum by translating natural-language privacy intents into cross-layer compute and network policies. It presents an LLM-driven architecture with a Knowledge Plane and an Orchestration Plane that convert intents into Kubernetes placements and ONOS routing rules, leveraging live state integration and a formal satisfaction relation $C ⊨_λ I$ to enforce compliance. A 90-intent benchmark and an automated validator assess end-to-end enforcement accuracy, scalability, and latency. Empirical results show GPT-4o achieving up to 95.6% accuracy with about 20–21 seconds of end-to-end latency, indicating practical viability for automated, privacy-aware orchestration across the cloud-edge continuum.

Abstract

With the rapid advancement of large language models (LLMs), efficiently serving LLM inference under limited GPU resources has become a critical challenge. Recently, an increasing number of studies have explored applying serverless computing paradigms to LLM serving in order to maximize resource utilization. However, LLM inference workloads are highly diverse, and modern GPU clusters are inherently heterogeneous, making it necessary to dynamically adjust deployment configurations online to better adapt to the elastic and dynamic nature of serverless environments. At the same time, enabling such online reconfiguration is particularly challenging due to the stateful nature of LLM inference and the massive size of model parameters. In this paper, we propose a dynamic pipeline reconfiguration approach that enables online adjustment of pipeline configurations while minimizing service downtime and performance degradation. Our method allows the system to select the optimal pipeline configuration in response to changing workloads. Experimental results on heterogeneous GPU platforms, including NVIDIA A100 and L40s, demonstrate that our migration mechanism incurs less than 50 ms of service downtime, while introducing under 10% overhead on both time-to-first-token (TTFT) and time-per-output-token (TPOT).

Figures (11)

• Figure 1: High-level architecture of the LLM-driven, intent-based privacy-preserving orchestration framework.
• Figure 2: System architecture for intent-driven privacy-aware orchestration.
• Figure 3: LLM-assisted service placement workflow.
• Figure 4: LLM-assisted network path control workflow.
• Figure 5: Example LLM-generated JSON constraints for ONOS.