Communication-Efficient Multi-Modal Edge Inference via Uncertainty-Aware Distributed Learning

TL;DR

The paper tackles distributed multi-modal edge inference over bandwidth-constrained wireless links by reducing training and inference communication while preserving robustness. It introduces a three-stage framework: Stage I performs fully local multi-modal self-supervised pre-training to disentangle shared and modality-specific representations; Stage II applies evidential deep learning with reliability-aware fusion for calibrated predictions; Stage III uses an uncertainty-guided retransmission policy to selectively gather additional features during inference. The approach is supported by information-theoretic guarantees for SSL under channel loss, a cross-/intra-modal pre-training loss, and a quantile-based uncertainty threshold to control communication, achieving higher accuracy with far fewer training rounds and robustness to channel variation and modality degradation. Experimental results on RGB-D indoor scene classification demonstrate significant training-efficiency gains, improved task performance, and robust inference under noisy or corrupted inputs, outperforming self-supervised and fully supervised baselines. The framework provides a practical pathway to efficient, robust edge intelligence in dynamic wireless environments with multi-modal data.

Abstract

Semantic communication is emerging as a key enabler for distributed edge intelligence due to its capability to convey task-relevant meaning. However, achieving communication-efficient training and robust inference over wireless links remains challenging. This challenge is further exacerbated for multi-modal edge inference (MMEI) by two factors: 1) prohibitive communication overhead for distributed learning over bandwidth-limited wireless links, due to the \emph{multi-modal} nature of the system; and 2) limited robustness under varying channels and noisy multi-modal inputs. In this paper, we propose a three-stage communication-aware distributed learning framework to improve training and inference efficiency while maintaining robustness over wireless channels. In Stage~I, devices perform local multi-modal self-supervised learning to obtain shared and modality-specific encoders without device--server exchange, thereby reducing the communication cost. In Stage~II, distributed fine-tuning with centralized evidential fusion calibrates per-modality uncertainty and reliably aggregates features distorted by noise or channel fading. In Stage~III, an uncertainty-guided feedback mechanism selectively requests additional features for uncertain samples, optimizing the communication--accuracy tradeoff in the distributed setting. Experiments on RGB--depth indoor scene classification show that the proposed framework attains higher accuracy with far fewer training communication rounds and remains robust to modality degradation or channel variation, outperforming existing self-supervised and fully supervised baselines.

Figures (5)

• Figure 1: Multi-modal semantic communication for edge--server collaborative inference.
• Figure 2: Three-stage evidential self-supervised multi-modal semantic communication: (I) multi-modal self-supervised pre-training (no device–server communication), (II) uncertainty-aware supervised fine-tuning with reliable fusion, and (III) uncertainty-guided adaptive retransmission. Per-view heads output Dirichlet (evidential) distributions. Example shown for RGB and depth.
• Figure 3: Mutual information analysis of single-modal versus multi-modal pre-training.
• Figure 4: NYUDv2 test accuracy versus communication rounds with full and few labels.
• Figure 5: SUN RGB-D test accuracy versus communication rounds with full and few labels.

Theorems & Definitions (1)

• Theorem 1: Information-theoretic Guarantee for Multi-modal SSL