Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Verifiable Dropout: Turning Randomness into a Verifiable Claim

Kichang Lee, Sungmin Lee, Jaeho Jin, JeongGil Ko

TL;DR

This paper tackles the problem of auditing stochastic components in outsourced AI training, focusing on dropout as a concrete source of non-determinism that can be exploited for undetected manipulation. It introduces Verifiable Dropout (VDO), a mechanism that binds dropout randomness to a cryptographically verifiable seed and produces zero-knowledge proofs of correct mask generation and application, without revealing sensitive activations. The authors formalize an end-to-end protocol, implement a PyTorch-friendly prototype atop a zkVM backend, and evaluate determinism, proving overhead, scalability, and attack-detection capabilities. Results show that dropout randomness can be made deterministically verifiable across platforms and that proofs effectively detect seed, probability, or activation tampering, albeit with meaningful proving latency that motivates selective auditing and potential hardware acceleration. This work enables post-hoc auditing of stochastic steps, strengthening accountability in cloud ML supply chains while preserving privacy and offering a pathway toward more trustworthy outsourced AI workflows.

Abstract

Modern cloud-based AI training relies on extensive telemetry and logs to ensure accountability. While these audit trails enable retrospective inspection, they struggle to address the inherent non-determinism of deep learning. Stochastic operations, such as dropout, create an ambiguity surface where attackers can mask malicious manipulations as natural random variance, granting them plausible deniability. Consequently, existing logging mechanisms cannot verify whether stochastic values were generated and applied honestly without exposing sensitive training data. To close this integrity gap, we introduce Verifiable Dropout, a privacy-preserving mechanism based on zero-knowledge proofs. We treat stochasticity not as an excuse but as a verifiable claim. Our approach binds dropout masks to a deterministic, cryptographically verifiable seed and proves the correct execution of the dropout operation. This design enables users to audit the integrity of stochastic training steps post-hoc, ensuring that randomness was neither biased nor cherry-picked, while strictly preserving the confidentiality of the model and data.

Verifiable Dropout: Turning Randomness into a Verifiable Claim

TL;DR

This paper tackles the problem of auditing stochastic components in outsourced AI training, focusing on dropout as a concrete source of non-determinism that can be exploited for undetected manipulation. It introduces Verifiable Dropout (VDO), a mechanism that binds dropout randomness to a cryptographically verifiable seed and produces zero-knowledge proofs of correct mask generation and application, without revealing sensitive activations. The authors formalize an end-to-end protocol, implement a PyTorch-friendly prototype atop a zkVM backend, and evaluate determinism, proving overhead, scalability, and attack-detection capabilities. Results show that dropout randomness can be made deterministically verifiable across platforms and that proofs effectively detect seed, probability, or activation tampering, albeit with meaningful proving latency that motivates selective auditing and potential hardware acceleration. This work enables post-hoc auditing of stochastic steps, strengthening accountability in cloud ML supply chains while preserving privacy and offering a pathway toward more trustworthy outsourced AI workflows.

Abstract

Modern cloud-based AI training relies on extensive telemetry and logs to ensure accountability. While these audit trails enable retrospective inspection, they struggle to address the inherent non-determinism of deep learning. Stochastic operations, such as dropout, create an ambiguity surface where attackers can mask malicious manipulations as natural random variance, granting them plausible deniability. Consequently, existing logging mechanisms cannot verify whether stochastic values were generated and applied honestly without exposing sensitive training data. To close this integrity gap, we introduce Verifiable Dropout, a privacy-preserving mechanism based on zero-knowledge proofs. We treat stochasticity not as an excuse but as a verifiable claim. Our approach binds dropout masks to a deterministic, cryptographically verifiable seed and proves the correct execution of the dropout operation. This design enables users to audit the integrity of stochastic training steps post-hoc, ensuring that randomness was neither biased nor cherry-picked, while strictly preserving the confidentiality of the model and data.
Paper Structure (30 sections, 6 figures, 1 table)

This paper contains 30 sections, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Attacks on Stochastic Training Components
  4. Verifiable Random Functions
  5. Proofs for Verifiable Computation and ML
  6. System Design
  7. Design Goals and Threat Model
  8. Goals.
  9. Threat model.
  10. System Overview
  11. Verifiable Random Seed Derivation
  12. Context binding.
  13. Evaluation.
  14. Why signatures.
  15. Platform-Stable PRG and Mask Generation
  16. ...and 15 more sections

Figures (6)

  • Figure 1: Deterministic vs. stochastic computation and the "plausible deniability" gap.
  • Figure 2: Overview workflow of VDO.
  • Figure 3: Local reference vs. zkVM commitments across shapes and dropout $p$.
  • Figure 4: Average runtime vs. dropout $p$ for each shape.
  • Figure 5: Per-forward time for Baseline dropout, Hash-only, and VDO.
  • ...and 1 more figures