Information-theoretic convergence of extreme values to the Gumbel distribution
Oliver Johnson
TL;DR
This work develops an information-theoretic perspective on convergence to the Gumbel distribution in extreme value theory by introducing a max-score $\Theta_X$ that linearizes under the maximum operation. It shows that the entropy satisfies $H(X)=1-\mathbb{E}\Theta_X(X)$ and derives a tractable relative-entropy expression to a Gumbel $Y$, enabling a strong convergence analysis. By studying the expected max-score of the standardized maximum $N_n=(M_n-b_n)/a_n$ and connecting it to the von Mises representation, the authors prove that $D(N_n\|Y) \to 0$ under mild tail and moment conditions, establishing convergence in relative entropy to a standard Gumbel. This provides a natural, information-theoretic justification for Gumbel convergence and a framework for establishing stronger convergence results than total variation.
Abstract
We show how convergence to the Gumbel distribution in an extreme value setting can be understood in an information-theoretic sense. We introduce a new type of score function which behaves well under the maximum operation, and which implies simple expressions for entropy and relative entropy. We show that, assuming certain properties of the von Mises representation, convergence to the Gumbel can be proved in the strong sense of relative entropy.