Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Task-Oriented Communication with Out-of-Distribution Detection: An Information Bottleneck Framework

Hongru Li, Wentao Yu, Hengtao He, Jiawei Shao, Shenghui Song, Jun Zhang, Khaled B. Letaief

TL;DR

This work addresses the open-world challenge of out-of-distribution (OoD) data in task-oriented communication for edge inference. It extends the information bottleneck (IB) framework to a class-conditional formulation (CCIB), incorporating class-conditioned latent priors and a contrastive (triplet) separation loss to make in-distribution latent representations distinct from OoD ones, while preserving the rate-distortion tradeoff via a variational upper bound $\mathcal{L}_{VCCIB}$. The approach uses Gaussian latent models with epoch-adaptive priors and end-to-end training over a joint source-channel coding pipeline, enabling reliable OoD detection without requiring OoD retraining. Experimental results on CIFAR-10 with LSUN and Tiny ImageNet OoD data show improved OoD detection (AUROC improvements) and robust performance under both static and dynamic channel conditions, demonstrating practical value for edge-enabled, low-latency inference in open-world networks.

Abstract

Task-oriented communication is an emerging paradigm for next-generation communication networks, which extracts and transmits task-relevant information, instead of raw data, for downstream applications. Most existing deep learning (DL)-based task-oriented communication systems adopt a closed-world scenario, assuming either the same data distribution for training and testing, or the system could have access to a large out-of-distribution (OoD) dataset for retraining. However, in practical open-world scenarios, task-oriented communication systems need to handle unknown OoD data. Under such circumstances, the powerful approximation ability of learning methods may force the task-oriented communication systems to overfit the training data (i.e., in-distribution data) and provide overconfident judgments when encountering OoD data. Based on the information bottleneck (IB) framework, we propose a class conditional IB (CCIB) approach to address this problem in this paper, supported by information-theoretical insights. The idea is to extract distinguishable features from in-distribution data while keeping their compactness and informativeness. This is achieved by imposing the class conditional latent prior distribution and enforcing the latent of different classes to be far away from each other. Simulation results shall demonstrate that the proposed approach detects OoD data more efficiently than the baselines and state-of-the-art approaches, without compromising the rate-distortion tradeoff.

Task-Oriented Communication with Out-of-Distribution Detection: An Information Bottleneck Framework

TL;DR

This work addresses the open-world challenge of out-of-distribution (OoD) data in task-oriented communication for edge inference. It extends the information bottleneck (IB) framework to a class-conditional formulation (CCIB), incorporating class-conditioned latent priors and a contrastive (triplet) separation loss to make in-distribution latent representations distinct from OoD ones, while preserving the rate-distortion tradeoff via a variational upper bound . The approach uses Gaussian latent models with epoch-adaptive priors and end-to-end training over a joint source-channel coding pipeline, enabling reliable OoD detection without requiring OoD retraining. Experimental results on CIFAR-10 with LSUN and Tiny ImageNet OoD data show improved OoD detection (AUROC improvements) and robust performance under both static and dynamic channel conditions, demonstrating practical value for edge-enabled, low-latency inference in open-world networks.

Abstract

Task-oriented communication is an emerging paradigm for next-generation communication networks, which extracts and transmits task-relevant information, instead of raw data, for downstream applications. Most existing deep learning (DL)-based task-oriented communication systems adopt a closed-world scenario, assuming either the same data distribution for training and testing, or the system could have access to a large out-of-distribution (OoD) dataset for retraining. However, in practical open-world scenarios, task-oriented communication systems need to handle unknown OoD data. Under such circumstances, the powerful approximation ability of learning methods may force the task-oriented communication systems to overfit the training data (i.e., in-distribution data) and provide overconfident judgments when encountering OoD data. Based on the information bottleneck (IB) framework, we propose a class conditional IB (CCIB) approach to address this problem in this paper, supported by information-theoretical insights. The idea is to extract distinguishable features from in-distribution data while keeping their compactness and informativeness. This is achieved by imposing the class conditional latent prior distribution and enforcing the latent of different classes to be far away from each other. Simulation results shall demonstrate that the proposed approach detects OoD data more efficiently than the baselines and state-of-the-art approaches, without compromising the rate-distortion tradeoff.
Paper Structure (10 sections, 1 theorem, 10 equations, 5 figures)

This paper contains 10 sections, 1 theorem, 10 equations, 5 figures.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. A CCIB Approach For Task-Oriented Communication
  4. Proposed CCIB Scheme
  5. DNN Parameterization and Contrastive Learning-Based Separation Loss
  6. Performance Evaluation
  7. Experiments Setup
  8. Experiments Results
  9. Conclusions
  10. Proof of Theorem 1

Key Result

Theorem 1

The variational upper bound of (eq:proposed_opt) is given by where $p_{\varphi}(\mathbf{\hat{z}}|\mathbf{x})$ is the conditional latent distribution given $(\mathbf{x}, y)$. $r(\mathbf{\hat{z}}|y)$ and $q_{\psi}(y|\mathbf{\hat{z}})$ are the variational approximation of the conditional latent prior $p(\mathbf{\hat{z}}|y)$ and $p(y|\mathbf{\hat{z}})$, repective

Figures (5)

  • Figure 1: The framework of the proposed CCIB-based task-oriented communication system.
  • Figure 2: The Venn diagrams of IB and CCIB show optimization (maximum and minimum) areas in their objectives. Blue lines: the area being minimized by both objectives. Gray lines: the area being maximized by both objective. The colored arrows indicate the information loss because of the channel noise. IB: The term to be maximized, i.e., $I(\hat{Z};Y)$, is a subset of the term to be minimized, namely, $I(\hat{Z};X)$. CCIB: The term to be maximized, i.e., $I(\hat{Z};Y)$, is no longer a subset of the term to be minimized, namely, $I(\hat{Z};X|Y)$.
  • Figure 3: T-SNE latent space projection for ID data CIFAR10 and OoD data LSUN-resized in the classification task with PSNR = 20 dB, classification accuracy $\geq 93\%$ and latency $\leq$ 6.5ms.
  • Figure 4: The AUROC-latency curves in static channel conditions with CIFAR10 as training data
  • Figure 5: The AUROC performance curves in dynamic channel conditions with CIFAR10 as training data

Theorems & Definitions (1)

  • Theorem 1