Activation Functions Considered Harmful: Recovering Neural Network Weights through Controlled Channels

TL;DR

This paper exposes a practical side-channel vulnerability in confidential ML deployments on Intel SGX, showing that input-dependent memory access in activation functions enables precise recovery of neural network weights and biases through instruction-granular page traces. The authors develop a novel attack using SGX-Step, deploy three TensorFlow Microlite PoCs, and demonstrate complete recovery of first-layer parameters with a modest number of queries, plus partial recovery for deeper layers under certain conditions. They provide a broad ecosystem analysis of activation-function implementations and discuss mitigations, emphasizing that current TEEs do not inherently secure confidential ML workloads without careful, constant-time, side-channel-resistant coding. The findings have significant implications for the security of outsourced ML in untrusted environments and call for rigorous side-channel vetting of ML libraries and frameworks. The work also offers a path toward quantifying trade-offs between attack practicality and model complexity, motivating future defenses across TEEs and ML stacks.

Abstract

With high-stakes machine learning applications increasingly moving to untrusted end-user or cloud environments, safeguarding pre-trained model parameters becomes essential for protecting intellectual property and user privacy. Recent advancements in hardware-isolated enclaves, notably Intel SGX, hold the promise to secure the internal state of machine learning applications even against compromised operating systems. However, we show that privileged software adversaries can exploit input-dependent memory access patterns in common neural network activation functions to extract secret weights and biases from an SGX enclave. Our attack leverages the SGX-Step framework to obtain a noise-free, instruction-granular page-access trace. In a case study of an 11-input regression network using the Tensorflow Microlite library, we demonstrate complete recovery of all first-layer weights and biases, as well as partial recovery of parameters from deeper layers under specific conditions. Our novel attack technique requires only 20 queries per input per weight to obtain all first-layer weights and biases with an average absolute error of less than 1%, improving over prior model stealing attacks. Additionally, a broader ecosystem analysis reveals the widespread use of activation functions with input-dependent memory access patterns in popular machine learning frameworks (either directly or via underlying math libraries). Our findings highlight the limitations of deploying confidential models in SGX enclaves and emphasise the need for stricter side-channel validation of machine learning implementations, akin to the vetting efforts applied to secure cryptographic libraries.

Figures (8)

• Figure 1: Victim enclaves exhibiting tight memory-access patterns can be precisely interrupted at instruction-level granularity using SGX-Step allowing to extract deterministic page-access count traces.
• Figure 2: Visualization of expf() CPU instruction count regions by input. The Underflow and Overflow regions have different instruction counts: 18 and 17 respectively. Note also the symmetrical regions of Inner1, inner2, and Inner3.
• Figure 3: Visualization of a single 2-input neuron. We use $\Sigma$ in this paper to represent the sum of all $i_i * w_i$ and $b$ passed into an activation function.
• Figure 4: Inset of an SGX-Step instruction-granular page access trace showing the execution of the first layer of a NN with 16 neurons. Note that the intra-page instruction pointer (RIP) values (recovered by putting an enclave in debug mode) are displayed here only for reference purposes and are not used in our attack. We include them to show that knowing the current page provides enough information to interpret traces without precise RIP values.
• Figure 5: Note that it is also possible to do this without matrix inversion for greater numerical efficiency, e.g., using numpy.linalg.lstsq(W, act_inv(target)-b).