Task-Oriented Communication for Edge Video Analytics

TL;DR

This paper tackles bandwidth bottlenecks in edge video analytics by introducing TOCOM-TEM, a task-oriented communication framework that transmits only task-relevant features rather than raw video. It combines a deterministic information bottleneck based feature extractor, a temporal entropy model for feature encoding, and a spatial-temporal fusion module at the server to perform joint inference on multi-view data. Through variational training and continuous relaxation, the approach achieves favorable rate–distortion tradeoffs on two challenging tasks: multi-camera pedestrian occupancy prediction and multi-camera object detection, outperforming data-oriented baselines such as JPEG, WebP, AVC, and HEVC. The results demonstrate substantial communication savings and low-latency inference, highlighting the practical impact of integrating task-oriented design with temporal and spatial correlation modeling in edge video analytics.

Abstract

With the development of artificial intelligence (AI) techniques and the increasing popularity of camera-equipped devices, many edge video analytics applications are emerging, calling for the deployment of computation-intensive AI models at the network edge. Edge inference is a promising solution to move the computation-intensive workloads from low-end devices to a powerful edge server for video analytics, but the device-server communications will remain a bottleneck due to the limited bandwidth. This paper proposes a task-oriented communication framework for edge video analytics, where multiple devices collect the visual sensory data and transmit the informative features to an edge server for processing. To enable low-latency inference, this framework removes video redundancy in spatial and temporal domains and transmits minimal information that is essential for the downstream task, rather than reconstructing the videos at the edge server. Specifically, it extracts compact task-relevant features based on the deterministic information bottleneck (IB) principle, which characterizes a tradeoff between the informativeness of the features and the communication cost. As the features of consecutive frames are temporally correlated, we propose a temporal entropy model (TEM) to reduce the bitrate by taking the previous features as side information in feature encoding. To further improve the inference performance, we build a spatial-temporal fusion module at the server to integrate features of the current and previous frames for joint inference. Extensive experiments on video analytics tasks evidence that the proposed framework effectively encodes task-relevant information of video data and achieves a better rate-performance tradeoff than existing methods.

Figures (12)

• Figure 1: An example of edge video analytics for pedestrian occupancy prediction. The edge server takes consecutive video frames from multiple cameras as inputs to estimate the position of pedestrians.
• Figure 2: Probabilistic modeling for edge video analytics systems. At each time step $t$, edge devices extract the task-relevant features $\bm{z}_{t}^{(k)}$ from their inputs $\bm{x}_{t}^{(k)}$, adopt the temporal entropy model to encode the quantized feature $\hat{\bm{z}}_{t}^{(k)}$, and forward them to an edge server for further processing. A spatial-temporal fusion module is deployed at the edge server that jointly leverages the current received features $\hat{\bm{z}}_{t}^{(1:K)}$ as well as the previous features $\{ \hat{\bm{z}}_{t-1}^{(1:K)},\ldots, \hat{\bm{z}}_{t-\tau_{1}}^{(1:K)} \}$ to predict the target variable $\hat{\bm{y}}_{t}$.
• Figure 3: Task-relevant feature extraction. Our method trains the feature extractors with hierarchical entropy models and auxiliary predictors.
• Figure 4: Temporal entropy model (left) and spatial-temporal fusion module (right). Once the feature extractors have been optimized, the proposed method carries out two steps: (1) It trains temporal entropy models to reduce the communication overhead by exploiting the redundancy among consecutive features. (2) It constructs a spatial-temporal fusion module to enhance inference performance by jointly utilizing the current and previous features.
• Figure 5: The rate-performance curves of different methods in the multi-camera pedestrian occupancy prediction task.

Theorems & Definitions (1)

• Remark 1