Principled Performance Tunability in Operating System Kernels

TL;DR

Perf-consts in the Linux kernel encode brittle hardware/workload assumptions and are not tunable at runtime. KernelX introduces Scoped Indirect Execution (SIE) to enable safe, in-situ tuning of any perf-const on deployed kernels through a programmable policy plane built on eBPF, without recompilation. The approach uses offline symbolic analysis to identify critical spans and derive symbolic state expressions, then synthesizes state-updating indirections that reflect new perf-const values while ensuring safe transitions via safe spans. Evaluations across 140 perf-consts show millisecond-scale policy updates, negligible overhead, and substantial performance improvements across multiple subsystems, including up to $50\times$ throughput gains in microbenchmarks and notable reductions in real-world workloads.

Abstract

The Linux kernel source code contains numerous constant values that critically influence system performance. Many of these constants, which we term perf-consts, are magic numbers that encode brittle assumptions about hardware and workloads. As systems and workloads evolve, such constants often become suboptimal. Unfortunately, deployed kernels lack support for safe and efficient in-situ tuning of perf-consts without a long and disruptive process of rebuilding and redeploying the kernel image. This paper advocates principled OS performance tunability. We present KernelX, a system that provides a safe, efficient, and programmable interface for in-situ tuning of arbitrary perf-consts on a running kernel. KernelX transforms any perf-const into a tunable knob on demand using a novel mechanism called Scoped Indirect Execution (SIE). SIE precisely identifies the binary boundaries where a perf-const influences system state and redirects execution to synthesized instructions that update the state as if new values were used. KernelX goes beyond version atomicity to guarantee side-effect safety, a property not provided by existing kernel update mechanisms. KernelX also provides a programmable interface that allows policies to incorporate application hints, hardware heuristics, and fine-grained isolation, without modifying kernel source code or disrupting deployed OS kernels. Case studies across multiple kernel subsystems demonstrate that KernelX enables significant performance improvements by making previously untunable perf-consts safely tunable at runtime, while supporting millisecond-scale policy updates.

Figures (16)

• Figure 1: Performance benefits of changing the value of a perf-const BLK_MAX_REQUEST_COUNT based on hardware devices and workloads. The default value is 32.
• Figure 2: Overview of KernelX.
• Figure 3: Symbolic expression derived from a perf-const($V$)'s seed instructions and its critical span. (a) and (b) show instructions from binaries rebuilt with $V'=10$ and $V"=17$. The seed is obtained by diffing (a) and (b). The $IV$ values in the symbolic expression are 10, 20, and 34.
• Figure 4: Synthesized SIE indirections for various CSes. An SIE indirection contains pairs of {location, update}. Locations are shown by the blue arrows.
• Figure 5: An example of a perf-const's CSes and SSes. Binary representation: SS1[118b, 1199], SS2[11be, 11c8].