Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latency-Distortion Tradeoffs in Communicating Classification Results over Noisy Channels

Noel Teku, Sudarshan Adiga, Ravi Tandon

TL;DR

This work addresses the problem of transmitting classifier decisions, represented as a probability vector $\mathbf{p}$, over noisy channels under a latency constraint. It introduces a latency-distortion framework based on $f$-divergence, and analyzes three quantization schemes—Uniform (UQ), Lattice (LQ), and Sparse Lattice (SLQ)—to bound source distortion and bit budgets. By combining these bounds with finite-blocklength channel coding results, the authors derive end-to-end latency formulas and optimize over the source distortion $\beta_s$ to meet a total distortion target $\beta_t$, including extensions to fading channels with/without CSI. Experiments on AWGN and Rayleigh fading channels, using datasets like CIFAR-100 and Imagenet-1K, show that SLQ consistently minimizes latency, especially for high-dimensional probability vectors, achieving up to large fractional reductions compared to UQ/LQ. The findings underscore the importance of joint source-channel design for low-latency semantic communications in ML-enabled systems.

Abstract

In this work, the problem of communicating decisions of a classifier over a noisy channel is considered. With machine learning based models being used in variety of time-sensitive applications, transmission of these decisions in a reliable and timely manner is of significant importance. To this end, we study the scenario where a probability vector (representing the decisions of a classifier) at the transmitter, needs to be transmitted over a noisy channel. Assuming that the distortion between the original probability vector and the reconstructed one at the receiver is measured via f-divergence, we study the trade-off between transmission latency and the distortion. We completely analyze this trade-off using uniform, lattice, and sparse lattice-based quantization techniques to encode the probability vector by first characterizing bit budgets for each technique given a requirement on the allowed source distortion. These bounds are then combined with results from finite-blocklength literature to provide a framework for analyzing the effects of both quantization distortion and distortion due to decoding error probability (i.e., channel effects) on the incurred transmission latency. Our results show that there is an interesting interplay between source distortion (i.e., distortion for the probability vector measured via f-divergence) and the subsequent channel encoding/decoding parameters; and indicate that a joint design of these parameters is crucial to navigate the latency-distortion tradeoff. We study the impact of changing different parameters (e.g. number of classes, SNR, source distortion) on the latency-distortion tradeoff and perform experiments on AWGN and fading channels. Our results indicate that sparse lattice-based quantization is the most effective at minimizing latency across various regimes and for sparse, high-dimensional probability vectors (i.e., high number of classes).

Latency-Distortion Tradeoffs in Communicating Classification Results over Noisy Channels

TL;DR

This work addresses the problem of transmitting classifier decisions, represented as a probability vector , over noisy channels under a latency constraint. It introduces a latency-distortion framework based on -divergence, and analyzes three quantization schemes—Uniform (UQ), Lattice (LQ), and Sparse Lattice (SLQ)—to bound source distortion and bit budgets. By combining these bounds with finite-blocklength channel coding results, the authors derive end-to-end latency formulas and optimize over the source distortion to meet a total distortion target , including extensions to fading channels with/without CSI. Experiments on AWGN and Rayleigh fading channels, using datasets like CIFAR-100 and Imagenet-1K, show that SLQ consistently minimizes latency, especially for high-dimensional probability vectors, achieving up to large fractional reductions compared to UQ/LQ. The findings underscore the importance of joint source-channel design for low-latency semantic communications in ML-enabled systems.

Abstract

In this work, the problem of communicating decisions of a classifier over a noisy channel is considered. With machine learning based models being used in variety of time-sensitive applications, transmission of these decisions in a reliable and timely manner is of significant importance. To this end, we study the scenario where a probability vector (representing the decisions of a classifier) at the transmitter, needs to be transmitted over a noisy channel. Assuming that the distortion between the original probability vector and the reconstructed one at the receiver is measured via f-divergence, we study the trade-off between transmission latency and the distortion. We completely analyze this trade-off using uniform, lattice, and sparse lattice-based quantization techniques to encode the probability vector by first characterizing bit budgets for each technique given a requirement on the allowed source distortion. These bounds are then combined with results from finite-blocklength literature to provide a framework for analyzing the effects of both quantization distortion and distortion due to decoding error probability (i.e., channel effects) on the incurred transmission latency. Our results show that there is an interesting interplay between source distortion (i.e., distortion for the probability vector measured via f-divergence) and the subsequent channel encoding/decoding parameters; and indicate that a joint design of these parameters is crucial to navigate the latency-distortion tradeoff. We study the impact of changing different parameters (e.g. number of classes, SNR, source distortion) on the latency-distortion tradeoff and perform experiments on AWGN and fading channels. Our results indicate that sparse lattice-based quantization is the most effective at minimizing latency across various regimes and for sparse, high-dimensional probability vectors (i.e., high number of classes).
Paper Structure (13 sections, 8 theorems, 43 equations, 7 figures, 1 algorithm)

This paper contains 13 sections, 8 theorems, 43 equations, 7 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. Main Results & Discussion
  4. Quantizing Classifier Probabilities
  5. Uniform Quantization (UQ)
  6. Lattice-based Quantization (LQ)
  7. Sparse Lattice-based Quantization (SLQ)
  8. Tradeoff Between Latency & End-to-End Distortion
  9. Generalization to Fading Channels
  10. Experimental Results
  11. Comparison of Quantization techniques
  12. Application to Fading Channels
  13. Conclusion

Key Result

Lemma 1

$T^{*}(\beta_{\text{max}})$ is convex non-increasing function of $\beta_{\text{max}}$. $D^{*}(T_\text{max})$ is convex non-increasing function of $T_\text{max}$.

Figures (7)

  • Figure 1: End-to-End block diagram for communicating classifier decisions (probability vector) over a noisy channel.
  • Figure 2: (a) Table summarizing the pre-trained network architectures and datasets used to generate Figure \ref{['fig:cifar200second']}. (b) Percentage of average mass of $k_\text{top}$ highest predictions from different pre-trained networks and datasets across $10,000$ test images. (c) Bit requirements for UQ, LQ, and SLQ (with $k_\text{top}= 5$ & $\delta = 0.00001$) for different class sizes as a function of source distortion.
  • Figure 3: Average mass of $k - k_\text{top}$ lowest probabilities for different values of $k_\text{top}$ for classifications made on 10,000 images of (a) CIFAR-100 (b) Imagenet-1K.
  • Figure 4: Lower convex hull of latencies for different $\beta_t$ for UQ, LQ, and SLQ (obtained from Theorem \ref{['thm1']}). Results are reported for $k = 10 (a), 50 (b)$ & $100 (c)$ to observe the impact varying the number of classes has on the quantization schemes.
  • Figure 5: Impact of varying source distortion on incurring latency for fixed $\beta_t = 0.05,0.2$ & $0.4$. Collecting results for (a) UQ, (b) LQ, (c) and SLQ assuming $k = 70$ & $k_{top} = 20$. It can be observed that as $\beta_t$ increases, the source distortion that obtains the minimal latency also increases. It can also be observed that SLQ is able to obtain the lowest latencies out of the three techniques.
  • ...and 2 more figures

Theorems & Definitions (13)

  • Lemma 1
  • Lemma 2
  • Remark 1
  • Lemma 3
  • Remark 2
  • Lemma 4
  • Remark 3
  • Lemma 5
  • Remark 4
  • Theorem 1
  • ...and 3 more