Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Information Bottleneck Theory for Multi-Modal Task-Aware Semantic Communication

Yujie Zhou, Cheng Peng, Rulong Wang, Yong Xiao, Yingyu Li, Guangming Shi, Ping Zhang

TL;DR

This work introduces a task-aware distributed information bottleneck (TADIB) framework to intelligently select and compress data modalities for multi-modal, multi-task semantic communication under resource constraints. By formulating a task-modality score and relaxing discrete modality-task selection into a probabilistic, cooperative policy (pTADIB), the authors enable end-to-end optimization of both modality selection and semantic codecs using variational bounds and gradient-based methods. The approach achieves comparable or better task inference with significantly reduced communication and computation across two public datasets, while providing theoretical guarantees on convergence and optimality under realistic constraints. The practical impact lies in enabling efficient, scalable, and task-tailored semantic communication for next-generation networks like 6G, where heterogeneous modalities and diverse tasks coexist at distributed devices.

Abstract

Semantic communication shifts the focus from bit-level accuracy to task-relevant semantic delivery, enabling efficient and intelligent communication for next-generation networks. However, existing multi-modal solutions often process all available data modalities indiscriminately, ignoring that their contributions to downstream tasks are often unequal. This not only leads to severe resource inefficiency but also degrades task inference performance due to irrelevant or redundant information. To tackle this issue, we propose a novel task-aware distributed information bottleneck (TADIB) framework, which quantifies the contribution of any set of modalities to given tasks. Based on this theoretical framework, we design a practical coding scheme that intelligently selects and compresses only the most task-relevant modalities at the transmitter. To find the optimal selection and the codecs in the network, we adopt the probabilistic relaxation of discrete selection, enabling distributed encoders to make coordinated decisions with score function estimation and common randomness. Extensive experiments on public datasets demonstrate that our solution matches or surpasses the inference quality of full-modal baselines while significantly reducing communication and computational costs.

Distributed Information Bottleneck Theory for Multi-Modal Task-Aware Semantic Communication

TL;DR

This work introduces a task-aware distributed information bottleneck (TADIB) framework to intelligently select and compress data modalities for multi-modal, multi-task semantic communication under resource constraints. By formulating a task-modality score and relaxing discrete modality-task selection into a probabilistic, cooperative policy (pTADIB), the authors enable end-to-end optimization of both modality selection and semantic codecs using variational bounds and gradient-based methods. The approach achieves comparable or better task inference with significantly reduced communication and computation across two public datasets, while providing theoretical guarantees on convergence and optimality under realistic constraints. The practical impact lies in enabling efficient, scalable, and task-tailored semantic communication for next-generation networks like 6G, where heterogeneous modalities and diverse tasks coexist at distributed devices.

Abstract

Semantic communication shifts the focus from bit-level accuracy to task-relevant semantic delivery, enabling efficient and intelligent communication for next-generation networks. However, existing multi-modal solutions often process all available data modalities indiscriminately, ignoring that their contributions to downstream tasks are often unequal. This not only leads to severe resource inefficiency but also degrades task inference performance due to irrelevant or redundant information. To tackle this issue, we propose a novel task-aware distributed information bottleneck (TADIB) framework, which quantifies the contribution of any set of modalities to given tasks. Based on this theoretical framework, we design a practical coding scheme that intelligently selects and compresses only the most task-relevant modalities at the transmitter. To find the optimal selection and the codecs in the network, we adopt the probabilistic relaxation of discrete selection, enabling distributed encoders to make coordinated decisions with score function estimation and common randomness. Extensive experiments on public datasets demonstrate that our solution matches or surpasses the inference quality of full-modal baselines while significantly reducing communication and computational costs.

Paper Structure

This paper contains 50 sections, 7 theorems, 44 equations, 14 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Backgrounds and Preliminaries
  4. Distributed Information Bottleneck
  5. Development Sequence
  6. DIB for Semantic Communication
  7. A Quick Review of DIB
  8. Notations
  9. System Model and Problem Formulation
  10. Task-aware Modality Contribution Metric
  11. General Definition of Task-Modality Score
  12. Applying the Score for Multi-modal Multi-task Dataset
  13. In-Network Task-aware Modality Selection
  14. Network Setting
  15. Finding the Optimal Set in Probability
  16. ...and 35 more sections

Key Result

Proposition 1

For a set of modality-task links $\mathcal{M}(\mathcal{T})$ w.r.t. $\upsilon$ on datasets $\{\mathcal{D}_t\}_{t\in \mathcal{T}}$, its corresponding task-modality score $\mathcal{R} \left(\mathcal{M}(\mathcal{T})\right)$ is empirically estimate by where parameters $\phi:= (\phi_t)_t$, $\varphi:= (\varphi_t)_t$, $\psi:= (\psi_t)_t$ are corresponding to optimizable codec $f$ and $g$ defined in (obj_

Figures (14)

  • Figure 1: Performance improvement and computational overhead w.r.t. modalities on MM-Fi. The more modalities involved (from left to right), the higher the computational cost. However, the corresponding inference performance does not increase monotonically when redundant/difficult-to-learn/irrelevant modalities (e.g., RGB image and LiDAR) are added.
  • Figure 2: Overview of a general multi-modal task-aware semantic communication system.
  • Figure 3: Task-aware modality selection in general distributed networks. Modalities (tasks) are distributed at different transmitters (receivers).
  • Figure 4: Rate-relevance tradeoff (sum-rate vs N-CE).
  • Figure 5: Training dynamics on information plane (sum-rate vs N-CE).
  • ...and 9 more figures

Theorems & Definitions (29)

  • Definition 1
  • Definition 2
  • Proposition 1
  • proof
  • Definition 3
  • Remark 1
  • Remark 2
  • Remark 3
  • Theorem 1
  • proof
  • ...and 19 more