Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Bregman Proximal Perspective on Classical and Quantum Blahut-Arimoto Algorithms

Kerry He, James Saunderson, Hamza Fawzi

TL;DR

This work unifies classical and quantum Blahut–Arimoto algorithms within a mirror-descent framework, showing BA iterations are entropic (and more generally Bregman) proximal updates for appropriately extended objectives. It leverages relative smoothness and relative strong convexity to recover sublinear and linear convergence rates, respectively, and extends the framework to problems with general linear constraints via primal-dual hybrid gradient (PDHG), including backtracking variants. By selecting kernel functions like entropic $-H$ and the negative log-determinant, the authors derive scalable algorithms for energy-constrained channel capacities, classical and quantum rate-distortion functions, and the relative entropy of quantum resources, all with provable convergence guarantees. The framework enables efficient, scalable solutions for large-scale information-theoretic problems and offers a clear path to extending BA-type methods to new constrained or resource-theoretic quantities in classical and quantum settings.

Abstract

The Blahut-Arimoto algorithm is a well-known method to compute classical channel capacities and rate-distortion functions. Recent works have extended this algorithm to compute various quantum analogs of these quantities. In this paper, we show how these Blahut-Arimoto algorithms are special instances of mirror descent, which is a type of Bregman proximal method, and a well-studied generalization of gradient descent for constrained convex optimization. Using recently developed convex analysis tools, we show how analysis based on relative smoothness and strong convexity recovers known sublinear and linear convergence rates for Blahut-Arimoto algorithms. This Bregman proximal viewpoint allows us to derive related algorithms with similar convergence guarantees to solve problems in information theory for which Blahut-Arimoto-type algorithms are not directly applicable. We apply this framework to compute energy-constrained classical and quantum channel capacities, classical and quantum rate-distortion functions, and approximations of the relative entropy of entanglement, all with provable convergence guarantees.

A Bregman Proximal Perspective on Classical and Quantum Blahut-Arimoto Algorithms

TL;DR

This work unifies classical and quantum Blahut–Arimoto algorithms within a mirror-descent framework, showing BA iterations are entropic (and more generally Bregman) proximal updates for appropriately extended objectives. It leverages relative smoothness and relative strong convexity to recover sublinear and linear convergence rates, respectively, and extends the framework to problems with general linear constraints via primal-dual hybrid gradient (PDHG), including backtracking variants. By selecting kernel functions like entropic and the negative log-determinant, the authors derive scalable algorithms for energy-constrained channel capacities, classical and quantum rate-distortion functions, and the relative entropy of quantum resources, all with provable convergence guarantees. The framework enables efficient, scalable solutions for large-scale information-theoretic problems and offers a clear path to extending BA-type methods to new constrained or resource-theoretic quantities in classical and quantum settings.

Abstract

The Blahut-Arimoto algorithm is a well-known method to compute classical channel capacities and rate-distortion functions. Recent works have extended this algorithm to compute various quantum analogs of these quantities. In this paper, we show how these Blahut-Arimoto algorithms are special instances of mirror descent, which is a type of Bregman proximal method, and a well-studied generalization of gradient descent for constrained convex optimization. Using recently developed convex analysis tools, we show how analysis based on relative smoothness and strong convexity recovers known sublinear and linear convergence rates for Blahut-Arimoto algorithms. This Bregman proximal viewpoint allows us to derive related algorithms with similar convergence guarantees to solve problems in information theory for which Blahut-Arimoto-type algorithms are not directly applicable. We apply this framework to compute energy-constrained classical and quantum channel capacities, classical and quantum rate-distortion functions, and approximations of the relative entropy of entanglement, all with provable convergence guarantees.
Paper Structure (19 sections, 20 theorems, 115 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 19 sections, 20 theorems, 115 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Main Results
  4. Preliminaries
  5. Notation
  6. Classical and Quantum Entropies
  7. Mirror Descent and Relative Smoothness
  8. Relative Smoothness and Strong Convexity
  9. Quantum Blahut-Arimoto Algorithms
  10. Equivalence with Mirror Descent
  11. Primal-Dual Hybrid Gradient
  12. Applications
  13. Energy-Constrained Channel Capacities
  14. Rate-Distortion Functions
  15. Relative Entropy of a Quantum Resource
  16. ...and 4 more sections

Key Result

Proposition 3.10

Let $f:\mathbb{V}\rightarrow\bar{\mathbb{R}}$ be a convex function such that $f$ is differentiable on $\mathop{\mathrm{int}}\nolimits\mathop{\mathrm{dom}}\nolimits f$. Let $\varphi:\mathbb{V}\rightarrow\bar{\mathbb{R}}$ be a Legendre function. Let $\mathcal{C}\subseteq\mathop{\mathrm{cl}}\nolimits\m Similarly, the following conditions are also equivalent:

Figures (3)

  • Figure 1: Convergence of PDHG to solve (a) the classical channel capacity with $n$ inputs and outputs and $l$ energy constraints, (b) the classical-quantum channel capacity with an alphabet size and channel input/output dimension of $n$ and $l$ energy constraints, and (c) the entanglement-assisted channel capacity with channel input/output dimension of $n$ and $l$ energy constraints.
  • Figure 2: Convergence of PDHG to solve (a) the classical rate-distortion for a Hamming distortion measure with $D=0.5$ and a channel with $n$ inputs and outputs, and (b) the quantum rate-distortion for an entanglement fidelity distortion measure with $D=0.5$ and a channel with input/output dimension of $n$.
  • Figure 3: Convergence of PDHG to compute the relative entropy of entanglement of density matrices with condition numbers $\kappa$ over PPT states for states defined over the tensor product of two Hilbert spaces with dimensions $n$ and $m$. Plots corresponding to bolded legend items correspond to problem instances in Table \ref{['tab:ree']}.

Theorems & Definitions (60)

  • Definition 3.1: Legendre function teboulle2018simplified
  • Definition 3.2: Bregman divergence bregman1967relaxation
  • Example 3.3
  • Example 3.4
  • Example 3.5
  • Example 3.6
  • Example 3.7
  • Definition 3.8: Relative smoothness
  • Definition 3.9: Relative strong convexity
  • Proposition 3.10
  • ...and 50 more