SILENT: A New Lens on Statistics in Software Timing Side Channels
Martin Dunsche, Patrick Bastian, Marcel Maehren, Nurullah Erinola, Robert Merget, Nicolai Bissantz, Holger Dette, Jörg Schwenk
TL;DR
This work targets the reliability gap in timing-side-channel analysis by introducing SILENT, a non-parametric, dependency-aware framework that provides strong statistical guarantees for timing measurements. It replaces exact-distribution tests with a Δ-negligibility approach and a bootstrap-based, quantile-focused max-test that handles dependent data and discrete measurements, enabling accurate control of false positives while delivering practical power analyses. The methodology yields actionable guidance for sample-size planning, tolerates negligible leaks per attacker models, and applies to real cryptographic and web contexts, with open-source tooling and artifacts. Collectively, SILENT enhances the adoption of timing-measurement validation in secure software development by delivering robust, interpretable, and configurable guarantees across realistic measurement environments.
Abstract
Cryptographic research takes software timing side channels seriously. Approaches to mitigate them include constant-time coding and techniques to enforce such practices. However, recent attacks like Meltdown [42], Spectre [37], and Hertzbleed [70] have challenged our understanding of what it means for code to execute in constant time on modern CPUs. To ensure that assumptions on the underlying hardware are correct and to create a complete feedback loop, developers should also perform \emph{timing measurements} as a final validation step to ensure the absence of exploitable side channels. Unfortunately, as highlighted by a recent study by Jancar et al. [30], developers often avoid measurements due to the perceived unreliability of the statistical analysis and its guarantees. In this work, we combat the view that statistical techniques only provide weak guarantees by introducing a new algorithm for the analysis of timing measurements with strong, formal statistical guarantees, giving developers a reliable analysis tool. Specifically, our algorithm (1) is non-parametric, making minimal assumptions about the underlying distribution and thus overcoming limitations of classical tests like the t-test, (2) handles unknown data dependencies in measurements, (3) can estimate in advance how many samples are needed to detect a leak of a given size, and (4) allows the definition of a negligible leak threshold $Δ$, ensuring that acceptable non-exploitable leaks do not trigger false positives, without compromising statistical soundness. We demonstrate the necessity, effectiveness, and benefits of our approach on both synthetic benchmarks and real-world applications.