A Unified Representation of Density-Power-Based Divergences Reducible to M-Estimation
Masahiro Kobayashi
TL;DR
This work introduces NB-DPD, a norm-based Bregman density power divergence with a flexible φ_γ and γ≥0, designed to yield estimators via M-estimation and to encompass classical divergences as special cases. By appropriate choices of φ_γ, NB-DPD recovers the density power divergence, γ-divergence, and the KL-divergence in the γ→0 limit, while enabling the construction of unified and new divergences through convex combinations. The paper analyzes the relationships to FDPD and Hölder-divergence, clarifies the intersections to BDPD and BHD, and demonstrates robustness properties, including a bounded influence function for γ>0 and a redescending property exclusive to γ-divergence. These results provide a flexible, theoretically grounded framework for robust estimation that connects and extends existing divergences, with practical implications for selecting φ_γ to trade off robustness and efficiency.
Abstract
Density-power-based divergences are known to provide robust inference procedures against outliers, and their extensions have been widely studied. A characteristic of successful divergences is that the estimation problem can be reduced to M-estimation. In this paper, we define a norm-based Bregman density power divergence (NB-DPD) -- density-power-based divergence with functional flexibility within the framework of Bregman divergences that can be reduced to M-estimation. We show that, by specifying the function $φ_γ$, NB-DPD reduces to well-known divergences, such as the density power divergence and the $γ$-divergence. Furthermore, by examining the combinations of functions $φ_γ$ corresponding to existing divergences, we show that a new divergence connecting these existing divergences can be derived. Finally, we show that the redescending property, one of the key indicators of robustness, holds only for the $γ$-divergence.