XLB: A High Performance Layer-7 Load Balancer for Microservices using eBPF-based In-kernel Interposition

TL;DR

XLB introduces an in-kernel L7 load balancer built with eBPF that interposes at the socket layer to eliminate the overhead of sidecar proxies in microservices. By relocating LB logic into the kernel and using novel socket types (p-sock and i-sock) and nested eBPF maps, XLB achieves higher throughput and substantially lower end-to-end latency than Istio and Cilium, while maintaining compatibility with existing Envoy control planes. The design emphasizes minimal data-path overhead, strong isolation, and compatibility, demonstrated through micro-benchmarks and real banking-scale deployments that show up to 1.5x throughput, 60% lower latency, and significant improvements in service density and chain scalability. Overall, XLB provides a practical, drop-in replacement for L7 LB in co-located microservice environments, enabling scalable performance without modifying applications or control-plane interfaces.

Abstract

L7 load balancers are a fundamental building block in microservices as they enable fine-grained traffic distribution. Compared to monolithic applications, microservices demand higher performance and stricter isolation from load balancers. This is due to the increased number of instances, longer service chains, and the necessity for co-location with services on the same host. Traditional sidecar-based load balancers are ill-equipped to meet these demands, often resulting in significant performance degradation. In this work, we present XLB, a novel architecture that reshapes L7 load balancers as in-kernel interposition operating on the socket layer. We leverage eBPF to implement the core load balancing logic in the kernel, and address the connection management and state maintenance challenges through novel socket layer redirection and nested eBPF maps designs. XLB eliminates the extra overhead of scheduling, communication, and data movement, resulting in a more lightweight, scalable, and efficient L7 load balancer architecture. Compared to the widely used microservices load balancers (Istio and Cilium), over 50 microservice instances, XLB achieves up to 1.5x higher throughput and 60% lower end-to-end latency.

Figures (12)

• Figure 1: Different architectures of various L7 LB designs. (a) represents Istio, which involves redundant networking stack processing and has a per-service proxy. (b) illustrates Cilium that optimizes away networking stack and uses a global proxy for all services. (c) implements the sidecar with DPDK to bypass the kernel. (d) implements the LB as a library within the application process. (e) is XLB, which offloads load-balancing logic into the kernel data path.
• Figure 2: Comparison of socket subsystem between the current kernel and XLB.
• Figure 3: A simplified example of converting Envoy configurations to C structures and eBPF maps. Configurations are in a tree hierarchy, and eBPF maps are organized using map-in-map structures.
• Figure 4: Simplified eBPF code for filter and route layer in Envoy configuration hierarchies (§\ref{['ss:state']}).
• Figure 5: Micro-benchmark results of various number of concurrent connections with zero size HTTP payload. All data are normalized to Istio single connection result.