Towards Reinforcement Learning for Exploration of Speculative Execution Vulnerabilities

TL;DR

This work addresses the challenge of detecting speculative execution vulnerabilities, such as Spectre, in post-silicon, black-box processors. It proposes SpecRL, a reinforcement-learning framework where an agent sequentially builds instruction sequences and receives observations from microarchitectural traces and counters, guided by a reward structure that favors detectable leaks. A case study on real hardware shows SpecRL finds leaks within about 7 minutes across program sizes and scales better than a fuzzing baseline, highlighting the method’s potential for scalable automated security analysis in microarchitectures. The paper discusses future enhancements, including expanding the instruction-action space, integrating input control, and leveraging distributed training to accelerate discovery of more complex leaks.

Abstract

Speculative attacks such as Spectre can leak secret information without being discovered by the operating system. Speculative execution vulnerabilities are finicky and deep in the sense that to exploit them, it requires intensive manual labor and intimate knowledge of the hardware. In this paper, we introduce SpecRL, a framework that utilizes reinforcement learning to find speculative execution leaks in post-silicon (black box) microprocessors.

Figures (2)

• Figure 1: SpecRL's training flow
• Figure 2: Average detection time as a function of program size for SpecRL and Revizor, using a simple instruction set.

Theorems & Definitions (1)

• Definition 1