Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A PAC-Bayesian Analysis of Channel-Induced Degradation in Edge Inference

Yangshuo He, Guanding Yu, Jingge Zhu

TL;DR

This work models edge inference where neural networks are partitioned across wireless edge devices, causing performance degradation due to channel distortion. By introducing an augmented network that treats the wireless channel as a stochastic l0 layer, the authors derive PAC-Bayesian generalization bounds that quantify the impact of channel-induced distortion on unseen channels. They develop channel-aware priors and a tractable training objective that integrates channel statistics into the learning process, yielding improved edge inference accuracy without full end-to-end retraining. Theoretical bounds are specialized to practical channels (BEC and Rayleigh), and simulations on MNIST and CIFAR-10 demonstrate that the proposed training scheme effectively mitigates the channel mismatch while providing interpretable bound components. Overall, the approach offers a principled framework for robust edge inference under wireless uncertainty and a practical training method leveraging channel statistics.

Abstract

In the emerging paradigm of edge inference, neural networks (NNs) are partitioned across distributed edge devices that collaboratively perform inference via wireless transmission. However, standard NNs are generally trained in a noiseless environment, creating a mismatch with the noisy channels during edge deployment. In this paper, we address this issue by characterizing the channel-induced performance deterioration as a generalization error against unseen channels. We introduce an augmented NN model that incorporates channel statistics directly into the weight space, allowing us to derive PAC-Bayesian generalization bounds that explicitly quantifies the impact of wireless distortion. We further provide closed-form expressions for practical channels to demonstrate the tractability of these bounds. Inspired by the theoretical results, we propose a channel-aware training algorithm that minimizes a surrogate objective based on the derived bound. Simulations show that the proposed algorithm can effectively improve inference accuracy by leveraging channel statistics, without end-to-end re-training.

A PAC-Bayesian Analysis of Channel-Induced Degradation in Edge Inference

TL;DR

This work models edge inference where neural networks are partitioned across wireless edge devices, causing performance degradation due to channel distortion. By introducing an augmented network that treats the wireless channel as a stochastic l0 layer, the authors derive PAC-Bayesian generalization bounds that quantify the impact of channel-induced distortion on unseen channels. They develop channel-aware priors and a tractable training objective that integrates channel statistics into the learning process, yielding improved edge inference accuracy without full end-to-end retraining. Theoretical bounds are specialized to practical channels (BEC and Rayleigh), and simulations on MNIST and CIFAR-10 demonstrate that the proposed training scheme effectively mitigates the channel mismatch while providing interpretable bound components. Overall, the approach offers a principled framework for robust edge inference under wireless uncertainty and a practical training method leveraging channel statistics.

Abstract

In the emerging paradigm of edge inference, neural networks (NNs) are partitioned across distributed edge devices that collaboratively perform inference via wireless transmission. However, standard NNs are generally trained in a noiseless environment, creating a mismatch with the noisy channels during edge deployment. In this paper, we address this issue by characterizing the channel-induced performance deterioration as a generalization error against unseen channels. We introduce an augmented NN model that incorporates channel statistics directly into the weight space, allowing us to derive PAC-Bayesian generalization bounds that explicitly quantifies the impact of wireless distortion. We further provide closed-form expressions for practical channels to demonstrate the tractability of these bounds. Inspired by the theoretical results, we propose a channel-aware training algorithm that minimizes a surrogate objective based on the derived bound. Simulations show that the proposed algorithm can effectively improve inference accuracy by leveraging channel statistics, without end-to-end re-training.
Paper Structure (10 sections, 4 theorems, 58 equations, 1 figure, 1 table)

This paper contains 10 sections, 4 theorems, 58 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. System Model and Augmented Network
  4. Probabilistic framework and Wireless Generalization Error
  5. Main Results
  6. Applications in Wireless Channels
  7. Wireless Channel Characterizations
  8. Generalization Bounds
  9. Channel-aware Training Algorithm
  10. Simulation

Key Result

Theorem 1

Assume that the loss function $l(w,z)$ is $\sigma$-sub-Gaussian under $P_Z$ for all $w\in\mathcal{W}$ and is $K$-Lipschitz continuous on $\mathcal{W}$ for all $z\in\mathcal{Z}$, then, for all data-independent prior distribution $Q_{W'W}$ over $\mathcal{W}\times\mathcal{W}$ and $k> 0$, w.p. at least where the distance metric $d_{\mathcal{W}}$ on the weight space is defined as the sum of layer-wise

Figures (1)

  • Figure 1: Left: the $L$-layer baseline NN, weights $\tilde{W}$ are given by the learning algorithm $P_{\tilde{W}|S}$. Middle: the augmented $L+1$-layer NN in noiseless training phase, where the additional $l_0$-th layer representing a wired connection. Weights $W$ are given by the learning algorithm $P_{W|S}$ defined in \ref{['equ:learning_algorithm']}. Right: the augmented $L+1$-layer NN deployed for edge inference, where the additional $l_0$-th layer representing a wireless channel. Weights $W'$ are given by the conditional distribution $P_{W'|S}$. The mismatch between the noiseless weights $W$ and the noisy weights $W'$ creates the generalization gap investigated in \ref{['equ:wireless_generalization_gap']}.

Theorems & Definitions (13)

  • Theorem 1
  • proof
  • Lemma 1: Donsker-Varadhan variational representation
  • Lemma 2: Markov's inequality
  • Corollary 1
  • proof
  • Remark 1
  • Example 1
  • proof
  • Example 2
  • ...and 3 more