Palladium: A DPU-enabled Multi-Tenant Serverless Cloud over Zero-copy Multi-node RDMA Fabrics

TL;DR

Palladium addresses the CPU-bound data plane in multi-tenant serverless clouds by offloading network data-plane tasks to a DPU-based engine (DNE) that cooperates with a host memory pool to achieve zero-copy inter-function communication. It combines cross-processor shared memory to avoid on-path CPU-DPU data moves, two-sided RDMA for lock-free inter-node transfers, and early transport adaptation at the cluster ingress to eliminate redundant protocol processing. The design provides per-tenant isolation and fairness for RNIC resources, while an edge ingress gateway terminates HTTP/TCP and routes payloads efficiently over RDMA. Empirical results show substantial improvements in throughput and latency (up to $20.9\times$ throughput, $21\times$ latency reduction) and CPU savings (up to $7$ CPU cores), demonstrating that DPU-led data planes can outperform CPU-based networks in serverless environments.

Abstract

Serverless computing promises enhanced resource efficiency and lower user costs, yet is burdened by a heavyweight, CPU-bound data plane. Prior efforts exploiting shared memory reduce overhead locally but fall short when scaling across nodes. Furthermore, serverless environments can have unpredictable and large-scale multi-tenancy, leading to contention for shared network resources. We present Palladium, a DPU-centric serverless data plane that reduces the CPU burden and enables efficient, zero-copy communication in multi-tenant serverless clouds. Despite the limited general-purpose processing capability of the DPU cores, Palladium strategically exploits the DPU's potential by (1) offloading data transmission to high-performance NIC cores via RDMA, combined with intra-node shared memory to eliminate data copies across nodes, and (2) enabling cross-processor (CPU-DPU) shared memory to eliminate redundant data movement, which overwhelms wimpy DPU cores. At the core of Palladium is the DPU-enabled network engine (DNE) -- a lightweight reverse proxy that isolates RDMA resources from tenant functions, orchestrates inter-node RDMA flows, and enforces fairness under contention. To further reduce CPU involvement, Palladium performs early HTTP/TCP-to-RDMA transport conversion at the cloud ingress, bridging the protocol mismatch before client traffic enters the RDMA fabric, thus avoiding costly protocol translation along the critical path. We show that careful selection of RDMA primitives (i.e., two-sided instead of one-sided) significantly affects the zero-copy data plane. Our preliminary experimental results show that enabling DPU offloading in Palladium improves RPS by 20.9x. The latency is reduced by a factor of 21x in the best case, all the while saving up to 7 CPU cores, and only consuming two wimpy DPU cores.

Figures (16)

• Figure 1: The architecture of serverless data plane (based on Knative's Serving/Eventing architecture knative).
• Figure 2: Selection of RDMA primitives for lock-free zero-copy serverless data plane.
• Figure 3: (1) On-path DPU offloading VS. (2) off-path DPU offloading (using cross-processor shared memory).
• Figure 4: Alternatives of transport protocol adaptation: (1) "Deferred" transport conversion within cluster; (2) Early transport conversion at cluster ingress (used by Palladium).
• Figure 5: A architecture-level overview of Palladium.