Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learning

Kotekar Annapoorna Prabhu, Andrew Gan, Zahra Ghodsi

TL;DR

This work identifies randomness as a critical but under-protected attack surface in machine learning. It analyzes how PRNGs are implemented and seeded across major frameworks and libraries, revealing variability and security gaps that attackers could exploit. The authors introduce RNGGuard, a two-phase system combining static policy generation with runtime enforcement to ensure cryptographically secure randomness and to detect tampering, supported by static and dynamic analyses. Through empirical evaluation in PyTorch and DP contexts, RNGGuard demonstrates practical protection against insecure RNG usage, with trade-offs in overhead that can be mitigated via asynchronous auditing and other optimizations, offering a tangible path to more trustworthy ML systems.

Abstract

Machine learning relies on randomness as a fundamental component in various steps such as data sampling, data augmentation, weight initialization, and optimization. Most machine learning frameworks use pseudorandom number generators as the source of randomness. However, variations in design choices and implementations across different frameworks, software dependencies, and hardware backends along with the lack of statistical validation can lead to previously unexplored attack vectors on machine learning systems. Such attacks on randomness sources can be extremely covert, and have a history of exploitation in real-world systems. In this work, we examine the role of randomness in the machine learning development pipeline from an adversarial point of view, and analyze the implementations of PRNGs in major machine learning frameworks. We present RNGGuard to help machine learning engineers secure their systems with low effort. RNGGuard statically analyzes a target library's source code and identifies instances of random functions and modules that use them. At runtime, RNGGuard enforces secure execution of random functions by replacing insecure function calls with RNGGuard's implementations that meet security specifications. Our evaluations show that RNGGuard presents a practical approach to close existing gaps in securing randomness sources in machine learning systems.

One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learning

TL;DR

This work identifies randomness as a critical but under-protected attack surface in machine learning. It analyzes how PRNGs are implemented and seeded across major frameworks and libraries, revealing variability and security gaps that attackers could exploit. The authors introduce RNGGuard, a two-phase system combining static policy generation with runtime enforcement to ensure cryptographically secure randomness and to detect tampering, supported by static and dynamic analyses. Through empirical evaluation in PyTorch and DP contexts, RNGGuard demonstrates practical protection against insecure RNG usage, with trade-offs in overhead that can be mitigated via asynchronous auditing and other optimizations, offering a tangible path to more trustworthy ML systems.

Abstract

Machine learning relies on randomness as a fundamental component in various steps such as data sampling, data augmentation, weight initialization, and optimization. Most machine learning frameworks use pseudorandom number generators as the source of randomness. However, variations in design choices and implementations across different frameworks, software dependencies, and hardware backends along with the lack of statistical validation can lead to previously unexplored attack vectors on machine learning systems. Such attacks on randomness sources can be extremely covert, and have a history of exploitation in real-world systems. In this work, we examine the role of randomness in the machine learning development pipeline from an adversarial point of view, and analyze the implementations of PRNGs in major machine learning frameworks. We present RNGGuard to help machine learning engineers secure their systems with low effort. RNGGuard statically analyzes a target library's source code and identifies instances of random functions and modules that use them. At runtime, RNGGuard enforces secure execution of random functions by replacing insecure function calls with RNGGuard's implementations that meet security specifications. Our evaluations show that RNGGuard presents a practical approach to close existing gaps in securing randomness sources in machine learning systems.
Paper Structure (32 sections, 3 figures, 6 tables)

This paper contains 32 sections, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Randomness in ML
  4. Pseudorandom Number Generators
  5. Randomness from a Distribution
  6. Distribution Statistical Tests
  7. Threat model
  8. Randomness in ML Systems
  9. Randomness Requirements and Attacker Objectives
  10. Ecosystem Study
  11. ML Frameworks
  12. ML-related Libraries
  13. DP Libraries
  14. Federated Learning (FL) Platforms
  15. Discussion
  16. ...and 17 more sections

Figures (3)

  • Figure 1: RNGGuard works in two phases - policy generation and runtime secure enforcement. Given a set of core RNG functions, RNGGuard identifies all functions in the target framework that depend on the core RNG functions using static analysis. At runtime, RNGGuard can operate in static and dynamic modes by enforces policies derived from our ML ecosystem survey and running statistical tests respectively.
  • Figure 2: Runtime of RNGGuard for one epoch training in non-DP and DP settings showing baseline (Base) and RNGGuard static (Stat) and with dynamic (Dyn) modes.
  • Figure 3: Runtime of RNGGuard for one epoch of DP training showing baseline (Base), No optimization (No op), as well as ASN and RASN optimizations.