Mitigating the Double-Reallocation Issue for IPv6 Lightweight Tunnel Encapsulations in the Linux Kernel
J. Iurman, E. Wansart, M. Goffart, B. Donnet
TL;DR
The paper tackles a CPU/NIC-dependent performance defect in Linux Lightweight Tunnels (LWTs) affecting Ioam, Srv6, and Rpl, where a double-reallocation of socket buffers can occur during encapsulation. It analyzes the root cause in headroom calculations and cache-line alignment, and introduces a kernel patch that uses a cached destination headroom via dst_dev_overhead to replace two reallocations with a single one. The authors validate the approach with Trex-based, line-rate tests, showing up to $28.8\%$ improvement for affected protocols and confirming safety for Rpl. The work provides a practical fix that restores forwarding efficiency across the affected protocols and hardens the kernel against similar patterns in the future.
Abstract
Lightweight Tunnels (LWTs) in the Linux kernel enable efficient per-route tunneling and are widely used by protocols such as In Situ Operations, Administration, and Maintenance (IOAM), Segment Routing over IPv6 (SRv6), and Routing Protocol for Low-Power and Lossy Networks (RPL). However, a performance issue was detected in their implementations, where a double-reallocation of socket buffers occurs under specific conditions, leading to significant throughput degradation. This paper investigates the root cause of the issue, which depends on the architecture of the Central Processing Unit (CPU) and the Network Interface Card (NIC). We propose a patch for the Linux kernel to fix this problem, replacing the double-reallocation with a single, efficient one. Performance evaluation demonstrates that the patch eliminates the inefficiency, improving forwarding rates by up to 28.8% for affected protocols.