Rényi Divergences in Central Limit Theorems: Old and New

TL;DR

This survey systematically analyzes Rényi and Tsallis divergences in the central limit theorem, highlighting when strong information-theoretic distances between sums of i.i.d. (and non-i.i.d.) variables and the Gaussian law converge to zero. It develops entropic CLTs, Edgeworth-type expansions, and non-uniform local limit theorems, linking convergence rates to moment and tail conditions, subgaussianity, and smoothness via Orlicz spaces and the Weierstrass transform. A central theme is the interplay between tail behavior, cumulants, and density regularity, with detailed results for D_α and D_∞ distances, including necessary and sufficient conditions and explicit rate bounds. The work further introduces and analyzes strictly subgaussian distributions, zeros of characteristic functions, and Esscher transforms, providing a comprehensive framework for CLTs under a broad class of strong divergence metrics and for multi-dimensional settings.

Abstract

We give an overview of various results and methods related to information-theoretic distances of Rényi type in the light of their applications to the central limit theorem (CLT). The first part (Sections 1-9) is devoted to the total variation and the Kullback-Leibler distance (relative entropy). In the second part (Sections 10-15) we discuss general properties of Rényi and Tsallis divergences of order $α>1$, and then in the third part (Sections 16-21) we turn to the CLT and non-uniform local limit theorems with respect to these strong distances. In the fourth part (Sections 22-31), we discuss recent results on strictly subgaussian distributions and describe necessary and sufficient conditions which ensure the validity of the CLT with respect to the Rényi divergence of infinite order.