QONNECT: A QoS-Aware Orchestration System for Distributed Kubernetes Clusters

TL;DR

The paper tackles QoS-aware cross-domain orchestration for applications spanning cloud, fog, and edge environments, addressing limitations of standard Kubernetes schedulers in honoring energy, cost, and latency constraints. It proposes QONNECT, a vendor-agnostic framework built on a Knowledge Base, Raft-based Resource Lead Agents, and per-cluster Resource Agents, translating declarative QoS goals into concrete placement and migration actions. A Borda-based scheduler computes a QoS-driven score, with the key relation $w = Energy_{Borda} \cdot q_E + Pricing_{Borda} \cdot q_P + Capacity_{Borda} \cdot q_C$ guiding decisions, and supports automated failover and dynamic redeployment. In a federated nine-cluster testbed, QONNECT demonstrates dynamic, policy-driven microservice placement, automated failover, and leader re-election without manual intervention, highlighting its potential to unify cloud–edge operations into a self-optimizing platform.

Abstract

Modern applications increasingly span across cloud, fog, and edge environments, demanding orchestration systems that can adapt to diverse deployment contexts while meeting Quality-of-Service (QoS) requirements. Standard Kubernetes schedulers do not account for user-defined objectives such as energy efficiency, cost optimization, and global performance, often leaving operators to make manual, cluster-by-cluster placement decisions. To address this need, we present QONNECT, a vendor-agnostic orchestration framework that enables declarative, QoS-driven application deployment across heterogeneous Kubernetes and K3s clusters. QONNECT introduces a distributed architecture composed of a central Knowledge Base, Raft-replicated Resource Lead Agents, and lightweight Resource Agents in each cluster. Through a minimal YAML-based interface, users specify high-level QoS goals, which the system translates into concrete placement and migration actions. Our implementation is evaluated on a federated testbed of up to nine cloud-fog-edge clusters using the Istio Bookinfo microservice application. The system demonstrates dynamic, policy-driven microservice placement, automated failover, QoS-compliant rescheduling, and leader re-election after node failure, all without manual intervention. By bridging the gap between declarative deployment models and operational QoS goals, QONNECT transforms the cloud-edge continuum into a unified, self-optimizing platform.

Figures (4)

• Figure 1: Target approach.
• Figure 2: Overview of Raft-based architecture components.
• Figure 3: Architecture overview: component interaction across cloud, fog, and edge domains.
• Figure 4: QONNECT prototype behavior in tests 1 to 3. The color specifies the corresponding test case.