Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hyper-VIB: A Hypernetwork-Enhanced Information Bottleneck Approach for Task-Oriented Communications

Jingchen Peng, Chaowen Deng, Yili Deng, Boxiang Ren, Lu Yang

TL;DR

Hyper-VIB introduces a hypernetwork-augmented information bottleneck framework for task-oriented communications in 6G, enabling joint device-network training that trades off task accuracy and communication overhead via a hyperparameter $\beta$. By deriving a tractable variational upper bound and employing a hypernetwork to map $\beta$ to optimal network parameters, Hyper-VIB achieves near-optimal performance with a single training run, substantially reducing computational cost compared to grid search. Theoretical analysis in the linear case demonstrates that the hypernetwork can represent the optimal response function, and experiments on image classification and wireless localization show improved training efficiency with competitive or superior accuracy relative to traditional VIB. This work offers a scalable, adaptable design for distributed AI over wireless links, with practical impact on 6G task-oriented system deployment.

Abstract

This paper presents Hyper-VIB, a hypernetwork-enhanced information bottleneck (IB) approach designed to enable efficient task-oriented communications in 6G collaborative intelligent systems. Leveraging IB theory, our approach enables an optimal end-to-end joint training of device and network models, in terms of the maximal task execution accuracy as well as the minimal communication overhead, through optimizing the trade-off hyperparameter. To address computational intractability in high-dimensional IB optimization, a tractable variational upper-bound approximation is derived. Unlike conventional grid or random search methods that require multiple training rounds with substantial computational costs, Hyper-VIB introduces a hypernetwork that generates approximately optimal DNN parameters for different values of the hyperparameter within a single training phase. Theoretical analysis in the linear case validates the hypernetwork design. Experimental results demonstrate our Hyper-VIB's superior accuracy and training efficiency over conventional VIB approaches in both classification and regression tasks.

Hyper-VIB: A Hypernetwork-Enhanced Information Bottleneck Approach for Task-Oriented Communications

TL;DR

Hyper-VIB introduces a hypernetwork-augmented information bottleneck framework for task-oriented communications in 6G, enabling joint device-network training that trades off task accuracy and communication overhead via a hyperparameter . By deriving a tractable variational upper bound and employing a hypernetwork to map to optimal network parameters, Hyper-VIB achieves near-optimal performance with a single training run, substantially reducing computational cost compared to grid search. Theoretical analysis in the linear case demonstrates that the hypernetwork can represent the optimal response function, and experiments on image classification and wireless localization show improved training efficiency with competitive or superior accuracy relative to traditional VIB. This work offers a scalable, adaptable design for distributed AI over wireless links, with practical impact on 6G task-oriented system deployment.

Abstract

This paper presents Hyper-VIB, a hypernetwork-enhanced information bottleneck (IB) approach designed to enable efficient task-oriented communications in 6G collaborative intelligent systems. Leveraging IB theory, our approach enables an optimal end-to-end joint training of device and network models, in terms of the maximal task execution accuracy as well as the minimal communication overhead, through optimizing the trade-off hyperparameter. To address computational intractability in high-dimensional IB optimization, a tractable variational upper-bound approximation is derived. Unlike conventional grid or random search methods that require multiple training rounds with substantial computational costs, Hyper-VIB introduces a hypernetwork that generates approximately optimal DNN parameters for different values of the hyperparameter within a single training phase. Theoretical analysis in the linear case validates the hypernetwork design. Experimental results demonstrate our Hyper-VIB's superior accuracy and training efficiency over conventional VIB approaches in both classification and regression tasks.

Paper Structure

This paper contains 15 sections, 1 theorem, 29 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. System Model and Data Transmission Chains
  4. Information Bottleneck (IB) Theory
  5. Variational Upper Bound of $\mathcal{C}_{\text{IB}}$
  6. The Hyper-VIB Approach
  7. Hypernetwork for Best Response
  8. Fully connected layers
  9. Convolutional layers
  10. Hyper-VIB objective and training algorithm design
  11. Optimality Analysis in the Linear Case
  12. Numerical Results and Discussions
  13. Experimental Settings
  14. Experimental Results
  15. Conclusion

Key Result

Theorem 1

Consider the linear model described in eq: liner-model. Given the network-side hypernetwork parameters $\boldsymbol{\psi}_{H}(\beta)$, there exist device-side parameters $\boldsymbol{\phi}_{H}(\beta)$ in our hypernetwork such that $\boldsymbol{\phi}_{H}(\beta) = \boldsymbol{\phi}^*(\beta)$ for all $

Figures (5)

  • Figure 1: The system model of device-network collaborative intelligence.
  • Figure 2: The hypernetwork framework with DNN and $\beta$ adjustment module in Hyper-VIB.
  • Figure 3: Layer-wise architecture of the hypernetwork.
  • Figure 4: Performance comparisons of our Hyper-VIB scheme with the conventional VIB benchmark, under varying $\beta$ values for different tasks.
  • Figure : Training Procedure for Hyper-VIB

Theorems & Definitions (2)

  • Theorem 1
  • proof