Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the sequential convergence of Lloyd's algorithms

Léo Portales, Elsa Cazelles, Edouard Pauwels

TL;DR

This work analyzes the sequential convergence of Lloyd-type algorithms for two semi-discrete optimal transport quantization problems: optimal quantization with arbitrary discrete weights and uniform quantization with equal weights. By viewing Lloyd updates as fixed-step gradient methods on the respective quantization losses, and assuming the target density is globally subanalytic (in particular analytic on compact sets), the authors prove convergence of the iterates to a critical point via the Kurdyka–Łojasiewicz framework, leveraging the log-analytic nature of the associated integrals. A key contribution is establishing the definability of the quantization losses in the o-minimal structure $\, eals_{ ext{an,exp}}$, which underpins the KL-based convergence and extends to broader semi-discrete OT losses, including general costs, sliced and max-sliced Wasserstein distances, and entropy-regularized losses. The results provide theoretical guarantees for widely used Lloyd-type quantization schemes and introduce powerful o-minimal techniques to semi-discrete OT, with potential extensions to convergence rate analysis and more general loss functionals.

Abstract

Lloyd's algorithm is an iterative method that solves the quantization problem, i.e. the approximation of a target probability measure by a discrete one, and is particularly used in digital applications.This algorithm can be interpreted as a gradient method on a certain quantization functional which is given by optimal transport. We study the sequential convergence (to a single accumulation point) for two variants of Lloyd's method: (i) optimal quantization with an arbitrary discrete measure and (ii) uniform quantization with a uniform discrete measure. For both cases, we prove sequential convergence of the iterates under an analiticity assumption on the density of the target measure. This includes for example analytic densities truncated to a compact semi-algebraic set. The argument leverages the log analytic nature of globally subanalytic integrals, the interpretation of Lloyd's method as a gradient method and the convergence analysis of gradient algorithms under Kurdyka-Lojasiewicz assumptions. As a by-product, we also obtain definability results for more general semi-discrete optimal transport losses such as transport distances with general costs, the max-sliced Wasserstein distance and the entropy regularized optimal transport loss.

On the sequential convergence of Lloyd's algorithms

TL;DR

This work analyzes the sequential convergence of Lloyd-type algorithms for two semi-discrete optimal transport quantization problems: optimal quantization with arbitrary discrete weights and uniform quantization with equal weights. By viewing Lloyd updates as fixed-step gradient methods on the respective quantization losses, and assuming the target density is globally subanalytic (in particular analytic on compact sets), the authors prove convergence of the iterates to a critical point via the Kurdyka–Łojasiewicz framework, leveraging the log-analytic nature of the associated integrals. A key contribution is establishing the definability of the quantization losses in the o-minimal structure , which underpins the KL-based convergence and extends to broader semi-discrete OT losses, including general costs, sliced and max-sliced Wasserstein distances, and entropy-regularized losses. The results provide theoretical guarantees for widely used Lloyd-type quantization schemes and introduce powerful o-minimal techniques to semi-discrete OT, with potential extensions to convergence rate analysis and more general loss functionals.

Abstract

Lloyd's algorithm is an iterative method that solves the quantization problem, i.e. the approximation of a target probability measure by a discrete one, and is particularly used in digital applications.This algorithm can be interpreted as a gradient method on a certain quantization functional which is given by optimal transport. We study the sequential convergence (to a single accumulation point) for two variants of Lloyd's method: (i) optimal quantization with an arbitrary discrete measure and (ii) uniform quantization with a uniform discrete measure. For both cases, we prove sequential convergence of the iterates under an analiticity assumption on the density of the target measure. This includes for example analytic densities truncated to a compact semi-algebraic set. The argument leverages the log analytic nature of globally subanalytic integrals, the interpretation of Lloyd's method as a gradient method and the convergence analysis of gradient algorithms under Kurdyka-Lojasiewicz assumptions. As a by-product, we also obtain definability results for more general semi-discrete optimal transport losses such as transport distances with general costs, the max-sliced Wasserstein distance and the entropy regularized optimal transport loss.
Paper Structure (35 sections, 23 theorems, 52 equations, 1 figure, 2 algorithms)

This paper contains 35 sections, 23 theorems, 52 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Lloyd-type algorithms.
  3. Convergence of gradient sequences and rigidity of semi-discrete optimal transport losses.
  4. Organization of the paper and notations.
  5. Main results
  6. Assumptions on the target measure
  7. Optimal Quantization
  8. Uniform Quantization
  9. Rigidity of quantization functionals and semi-discrete optimal transport losses
  10. Convergence of the iterates of Lloyd's algorithms
  11. KL inequality
  12. Lloyd's sequence for optimal quantization
  13. Lloyd's sequence for uniform quantization
  14. Rigidity of semi-discrete optimal transport quantization losses
  15. O-minimal structures
  16. ...and 20 more sections

Key Result

Theorem 2.3

Let $\mu$ be as in Assumption ass:convexSupport, then the iterates of Lloyd's algorithm for optimal quantization in Algorithm alg:LOYD_optimal_quant converge to a critical point of $G_N$ in eq:optimal_quant.

Figures (1)

  • Figure 1: (Left) Target Gaussian mixture $\mu$ with two components truncated on a disk. (Middle) Optimal quantization of $\mu$ with $20$ points (blue) and their corresponding Voronoi cells (in red) after $250$ iterations of Lloyd's algorithm. (Right) Uniform quantization of $\mu$ with $20$ points (blue) and the corresponding power cells (in red) after $5$ iterations of Lloyd's algorithm adjusted for uniform quantization. The diameter of a blue dot is proportional to its weight. The algorithms are randomly initialized and run using the PyMongeAmpere library\ref{['note1']}.

Theorems & Definitions (51)

  • Example 2.1
  • Remark 2.2
  • Theorem 2.3
  • Remark 2.4
  • Theorem 2.5
  • Remark 2.6
  • Definition 3.1: Definition 7 in proximalbolte2010defKL
  • Theorem 3.2: Theorem 1 in KurdykaarticleominimpliqueKL
  • Proposition 3.3: Proposition 6.2 in Lloydoptcvonedimlogconcave2
  • Proposition 3.4: Corollary 3.7 in nondegeneracyLloyd
  • ...and 41 more