SHADOW: Seamless Handoff And Zero-Downtime Orchestrated Workload Migration for Stateful Microservices

Abstract

Migrating stateful microservices in Kubernetes requires careful state management because in-memory state is lost when a container restarts. For StatefulSet-managed workloads, the problem is compounded by identity constraints that prohibit two pods with the same ordinal from running simultaneously, forcing a sequential stop-recreate cycle with a median 38.5s of service downtime. This paper presents SHADOW Seamless Handoff And Zero-Downtime Orchestrated Workload Migration, a Kubernetes-native framework that implements the Message-based Stateful Microservice Migration (MS2M) approach as a Kubernetes Operator. SHADOW introduces the ShadowPod strategy, where a shadow pod is created from a CRIU checkpoint image on the target node while the source pod continues serving traffic, allowing concurrent operation during message replay. For StatefulSet workloads, an identity swap procedure with the ExchangeFence mechanism re-checkpoints the shadow pod, creates a StatefulSet-owned replacement, and drains both message queues to guarantee zero message loss during the handoff. An evaluation on a bare-metal Kubernetes cluster with 280 migration runs across four configurations and seven message rates (10--120msg/s) shows that, compared to the sequential baseline on the same StatefulSet workload, the ShadowPod strategy reduces the restore phase by up to 92%, eliminates service downtime entirely, and reduces total migration time by up to 77%, with zero message loss across all 280 runs.

Figures (5)

• Figure 1: Architecture of the SHADOW framework. The reconciler watches StatefulMigration custom resources and orchestrates migration through the API server: checkpoint creation via the kubelet API (1), checkpoint transfer between ms2m-agent instances (direct) or via registry (2), shadow pod creation on the target node (3), and traffic switchover during finalization (4).
• Figure 2: State machine of the StatefulMigration reconciler. The reconcile loop dispatches to the handler for the current phase, advancing on success or transitioning to Failed on error. Each handler is idempotent for safe retry.
• Figure 3: Timeline comparison of the migration strategies. SS-Seq: service is unavailable during the 38.5 s restore phase. SS-Shadow/D-Reg: the source continues serving while the shadow pod restores and replays. SS-Swap: after replay, the shadow pod continues serving during the identity swap (re-checkpoint, ExchangeFence), and is replaced only once the StatefulSet-owned replacement is ready. All ShadowPod variants achieve zero downtime.
• Figure 4: Total migration time (median, $n=10$) across message rates. At low rates ($\leq$ 40 msg/s), ShadowPod reduces time by 73--76%. At high rates where the 120 s replay cutoff dominates, the reduction narrows to 22%. The dashed line marks the replay cutoff boundary.
• Figure 5: Phase duration breakdown at 60 msg/s (medians, $n=10$). The Sequential configuration is dominated by the 38.4 s restore phase. ShadowPod reduces restore to 2.5--2.9 s, shifting the bottleneck entirely to replay.