Adversarial Attacks to Latent Representations of Distributed Neural Networks in Split Computing

TL;DR

This work investigates the adversarial robustness of distributed DNNs under split computing by framing latent representations within an information-theoretic lens. Using the Information Bottleneck, it shows that robustness improves with deeper splitting points but may worsen if the latent dimension is too small due to bias, revealing a fundamental depth-dimension trade-off. The authors validate these insights through extensive experiments across six architectures, six distributed approaches, and ten attacks on ImageNet-1K, plus additional tasks, demonstrating that latent-space perturbations are consistently less effective than input-space ones, especially with bottlenecks and larger depths. The findings provide design guidance for robust, latency-aware distributed inference and open avenues for jointly optimizing accuracy, bandwidth, and adversarial resilience in edge-cloud systems.

Abstract

Distributed deep neural networks (DNNs) have been shown to reduce the computational burden of mobile devices and decrease the end-to-end inference latency in edge computing scenarios. While distributed DNNs have been studied, to the best of our knowledge, the resilience of distributed DNNs to adversarial action remains an open problem. In this paper, we fill the existing research gap by rigorously analyzing the robustness of distributed DNNs against adversarial action. We cast this problem in the context of information theory and rigorously proved that (i) the compressed latent dimension improves the robustness but also affect task-oriented performance; and (ii) the deeper splitting point enhances the robustness but also increases the computational burden. These two trade-offs provide a novel perspective to design robust distributed DNN. To test our theoretical findings, we perform extensive experimental analysis by considering 6 different DNN architectures, 6 different approaches for distributed DNN and 10 different adversarial attacks using the ImageNet-1K dataset.

Figures (7)

• Figure 1: Overview of Adversarial Attacks to Distributed .
• Figure 2: Threat model under consideration. (Left) The adversary plays a man-in-the-middle attack where the communication between mobile and local devices are altered without detection; (Right) Difference between adversarial attack in input space and adversarial attack in latent space.
• Figure 3: Modeling DNN with IB. Each representation $T_i$ only depends on the previous output $T_{i-1}$, and the optimal $T^{*}_{i}$ can be interpreted as the IB which optimizes \ref{['eqn:ib']} at layer $i$.
• Figure 4: 10 different attacks to ResNet152-fc with perturbation budget $\epsilon = 0.01$.
• Figure 5: Whitebox baseline (PGD) and blackbox attacks under $l_\infty$ and $l_2$ in input and latent space with perturbation budget $\epsilon = 0.003$ applied to 6 different .

Theorems & Definitions (9)

• Lemma 1: Latent Robustness
• proof
• Theorem 1: Compression-Robustness Trade-Off
• proof
• Lemma 2: Information Distortion
• proof
• Theorem 2: Input vs Latent Robustness
• proof
• Corollary 1: Depth-Robustness Trade-Off