Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information Compression in the AI Era: Recent Advances and Future Challenges

Jun Chen, Yong Fang, Ashish Khisti, Ayfer Ozgur, Nir Shlezinger, Chao Tian

TL;DR

The paper surveys how machine learning reshapes data compression, moving beyond classical rate-distortion theory to include task-based, goal-oriented quantization and rate-distortion-perception tradeoffs. It presents theory for GOQ, including linear and quadratic estimation tasks, and analyzes RD-Perception under no or ample common randomness, with extensions to estimation and learning in distributed settings. It then details AI-enabled approaches, including neural GOQ (both fixed and learned quantizers), LLM-based compression, and deep learning–based image/video transmission (e.g., DeepJSCC and variants), highlighting practical gains and challenges. The authors advocate a hybrid model-based and data-driven design philosophy, urging integration of interpretable GOQ with learning-based tools, and outlining future directions for scalable, robust, and perceptually aware compression in the AI era.

Abstract

This survey articles focuses on emerging connections between the fields of machine learning and data compression. While fundamental limits of classical (lossy) data compression are established using rate-distortion theory, the connections to machine learning have resulted in new theoretical analysis and application areas. We survey recent works on task-based and goal-oriented compression, the rate-distortion-perception theory and compression for estimation and inference. Deep learning based approaches also provide natural data-driven algorithmic approaches to compression. We survey recent works on applying deep learning techniques to task-based or goal-oriented compression, as well as image and video compression. We also discuss the potential use of large language models for text compression. We finally provide some directions for future research in this promising field.

Information Compression in the AI Era: Recent Advances and Future Challenges

TL;DR

The paper surveys how machine learning reshapes data compression, moving beyond classical rate-distortion theory to include task-based, goal-oriented quantization and rate-distortion-perception tradeoffs. It presents theory for GOQ, including linear and quadratic estimation tasks, and analyzes RD-Perception under no or ample common randomness, with extensions to estimation and learning in distributed settings. It then details AI-enabled approaches, including neural GOQ (both fixed and learned quantizers), LLM-based compression, and deep learning–based image/video transmission (e.g., DeepJSCC and variants), highlighting practical gains and challenges. The authors advocate a hybrid model-based and data-driven design philosophy, urging integration of interpretable GOQ with learning-based tools, and outlining future directions for scalable, robust, and perceptually aware compression in the AI era.

Abstract

This survey articles focuses on emerging connections between the fields of machine learning and data compression. While fundamental limits of classical (lossy) data compression are established using rate-distortion theory, the connections to machine learning have resulted in new theoretical analysis and application areas. We survey recent works on task-based and goal-oriented compression, the rate-distortion-perception theory and compression for estimation and inference. Deep learning based approaches also provide natural data-driven algorithmic approaches to compression. We survey recent works on applying deep learning techniques to task-based or goal-oriented compression, as well as image and video compression. We also discuss the potential use of large language models for text compression. We finally provide some directions for future research in this promising field.
Paper Structure (19 sections, 3 theorems, 29 equations, 6 figures)

This paper contains 19 sections, 3 theorems, 29 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Compression Theory Beyond Rate-Distortion
  3. Goal-Oriented Quantization
  4. High Level Rationale
  5. System Model
  6. Linear Estimation Tasks
  7. Quadratic Estimation Tasks
  8. Rate-Distortion-Perception Theory
  9. No Common Randomness
  10. Unlimited Common Randomness
  11. Perfect Realism
  12. Compression for Estimation and Learning
  13. AI-Enhanced Compression
  14. Neural Goal- Oriented Compression
  15. System Model
  16. ...and 4 more sections

Key Result

Lemma 2

Figures (6)

  • Figure 1: Quantizer illustration.
  • Figure 2: Block diagram of considered goal-oriented quantization systems.
  • Figure 3: Distributed estimation under communication constraints.
  • Figure 4: Neural goal-oriented quantization system with scalar quantizers illustration.
  • Figure 5: Separate design of source coding, channel coding, and modulation for an image/video communications system.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Definition 1: Quantizer
  • Lemma 2
  • Theorem 3
  • Theorem 4