Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QML-IB: Quantized Collaborative Intelligence between Multiple Devices and the Mobile Network

Jingchen Peng, Boxiang Ren, Lu Yang, Chenghui Peng, Panpan Niu, Hao Wu

TL;DR

The paper addresses efficient task-specific data transmission for collaborative AI across multiple mobile devices and a network, introducing the multi-link information bottleneck (ML-IB) framework and a quantized variant (QML-IB). It defines a computable objective $\mathcal{C}_{\text{ML-IB}}$ balancing task accuracy and communication overhead, then derives a variational upper bound and a Log-Sum Inequality-based approximation to enable training with quantized communications. The proposed QML-IB algorithm trains device encoders, a network decoder, and a trainable quantizer to achieve high accuracy with low transmission cost, validated on MNIST and CIFAR-10 against a state-of-the-art baseline. Results show robust performance under varying latency, device counts, and channel conditions, with quantization incursing only negligible performance loss and even improving results in overlapping-data scenarios. The approach is applicable beyond AWGN to other fading channels, highlighting its practical relevance for 6G-native intelligent networks.

Abstract

The integration of artificial intelligence (AI) and mobile networks is regarded as one of the most important scenarios for 6G. In 6G, a major objective is to realize the efficient transmission of task-relevant data. Then a key problem arises, how to design collaborative AI models for the device side and the network side, so that the transmitted data between the device and the network is efficient enough, which means the transmission overhead is low but the AI task result is accurate. In this paper, we propose the multi-link information bottleneck (ML-IB) scheme for such collaborative models design. We formulate our problem based on a novel performance metric, which can evaluate both task accuracy and transmission overhead. Then we introduce a quantizer that is adjustable in the quantization bit depth, amplitudes, and breakpoints. Given the infeasibility of calculating our proposed metric on high-dimensional data, we establish a variational upper bound for this metric. However, due to the incorporation of quantization, the closed form of the variational upper bound remains uncomputable. Hence, we employ the Log-Sum Inequality to derive an approximation and provide a theoretical guarantee. Based on this, we devise the quantized multi-link information bottleneck (QML-IB) algorithm for collaborative AI models generation. Finally, numerical experiments demonstrate the superior performance of our QML-IB algorithm compared to the state-of-the-art algorithm.

QML-IB: Quantized Collaborative Intelligence between Multiple Devices and the Mobile Network

TL;DR

The paper addresses efficient task-specific data transmission for collaborative AI across multiple mobile devices and a network, introducing the multi-link information bottleneck (ML-IB) framework and a quantized variant (QML-IB). It defines a computable objective balancing task accuracy and communication overhead, then derives a variational upper bound and a Log-Sum Inequality-based approximation to enable training with quantized communications. The proposed QML-IB algorithm trains device encoders, a network decoder, and a trainable quantizer to achieve high accuracy with low transmission cost, validated on MNIST and CIFAR-10 against a state-of-the-art baseline. Results show robust performance under varying latency, device counts, and channel conditions, with quantization incursing only negligible performance loss and even improving results in overlapping-data scenarios. The approach is applicable beyond AWGN to other fading channels, highlighting its practical relevance for 6G-native intelligent networks.

Abstract

The integration of artificial intelligence (AI) and mobile networks is regarded as one of the most important scenarios for 6G. In 6G, a major objective is to realize the efficient transmission of task-relevant data. Then a key problem arises, how to design collaborative AI models for the device side and the network side, so that the transmitted data between the device and the network is efficient enough, which means the transmission overhead is low but the AI task result is accurate. In this paper, we propose the multi-link information bottleneck (ML-IB) scheme for such collaborative models design. We formulate our problem based on a novel performance metric, which can evaluate both task accuracy and transmission overhead. Then we introduce a quantizer that is adjustable in the quantization bit depth, amplitudes, and breakpoints. Given the infeasibility of calculating our proposed metric on high-dimensional data, we establish a variational upper bound for this metric. However, due to the incorporation of quantization, the closed form of the variational upper bound remains uncomputable. Hence, we employ the Log-Sum Inequality to derive an approximation and provide a theoretical guarantee. Based on this, we devise the quantized multi-link information bottleneck (QML-IB) algorithm for collaborative AI models generation. Finally, numerical experiments demonstrate the superior performance of our QML-IB algorithm compared to the state-of-the-art algorithm.
Paper Structure (17 sections, 3 theorems, 35 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 17 sections, 3 theorems, 35 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. System Model and Data Transmission Chains
  4. The ML-IB Scheme
  5. Variational Upper Bound of $\mathcal{C}_{\text{ML-IB}}$
  6. Computable ML-IB Model
  7. Quantization Design
  8. Approximation of Variational Upper bound
  9. Numerical Results and Discussions
  10. Experimental Settings
  11. Experimental Results
  12. Conclusion
  13. Proof of Lemma \ref{['lemma: 1']}
  14. Proof of Theorem \ref{['theorem: 2']}
  15. Ablation Study
  16. ...and 2 more sections

Key Result

Lemma 1

The $D_{\text{KL}}^*$ defined in eq: upper KL serves as an upper bound for the KL divergence within $\hat{\mathcal{C}}_{\text{ML-IB}}$:

Figures (4)

  • Figure 1: System model of collaborative intelligence between multiple devices and the network.
  • Figure 2: Performance comparison between our algorithm and VDDIB-SR in different experimental settings. Evaluations on MNIST Dataset in Figs. \ref{['fig: experimental results']}(a), (b), (c) and on CIFAR-10 Dataset in Figs. \ref{['fig: experimental results']}(d), (e), (f).
  • Figure 3: Impact of the quantization bit depth on performance
  • Figure 4: Performance in partial overlap of original input data

Theorems & Definitions (3)

  • Lemma 1
  • Theorem 1
  • Theorem 2