Generalized Divergence Measures and Weak Convergence for the Sets of Probability Measures
Xinpeng Li, Miao Yu
TL;DR
This work extends classical information divergences to the setting of distributional uncertainty by introducing generalized KL and Jensen-Shannon divergences between sets of probability measures: $KL(P||Q)=sup_{μ∈P} inf_{ν∈Q} KL(μ||ν)$ and $JS(P,Q)=1/2 KL(P||M) + 1/2 KL(Q||M)$ with $M=(P+Q)/2$. It establishes a generalized duality formula, a Pinsker-type inequality, and a range of convergence results, including weak convergence under sublinear expectations, via a robust framework for distributional uncertainty. The analysis connects divergence-based convergence with total variation and Wasserstein-type metrics, showing that convergence in $ar{KL}$ implies convergence in $V$ and $JS$, and that weak convergence of sublinear expectations follows from these approximations (with $W_1$ convergence when the diameter is finite). These results provide a rigorous foundation for robust inference and decision-making under model and distributional uncertainty. Overall, the paper contributes a coherent methodology for comparing and converging sets of distributions in contexts where exact specification is infeasible.
Abstract
This paper extends the asymmetric Kullback-Leibler divergence and symmetric Jensen-Shannon divergence from two probability measures to the case of two sets of probability measures. We establish some fundamental properties of these generalized divergences, including a duality formula and a Pinsker-type inequality. Furthermore, convergence results are derived for both the generalized asymmetric and symmetric divergences, as well as for weak convergence under sublinear expectations.