Robust $M$-Estimation of Scatter Matrices via Precision Structure Shrinkage

TL;DR

This work introduces a robust M-estimator for scatter matrices that incorporates precision-structure shrinkage toward the identity to combat deterioration under clustered outliers in high dimensions. By embedding an EM-type update for the precision matrix into penalized M-estimation, the authors derive existence conditions, interpret the method within Bayesian and penalized frameworks, and establish AdBP bounds that improve robustness relative to classical estimators. Numerical experiments demonstrate that precision-shrinkage methods maintain accuracy where Tyler's M-estimator and LNSMI fail in the presence of clustered outliers, while preserving competitive performance in standard settings. The approach provides a principled, theoretically grounded way to enhance robustness in scatter estimation for elliptical models with unknown density generators, with practical guidance on coefficient selection and a clear link to existing shrinkage-based methods.

Abstract

Maronna's and Tyler's $M$-estimators are among the most widely used robust estimators for scatter matrices. However, when the dimension of observations is relatively high, their performance can substantially deteriorate in certain situations, particularly in the presence of clustered outliers. To address this issue, we propose an estimator that shrinks the estimated precision matrix toward the identity matrix. We derive a sufficient condition for its existence, discuss its statistical interpretation, and establish upper and lower bounds for its breakdown point. Numerical experiments confirm robustness of the proposed method.

Figures (8)

• Figure 1: An illustration of the distribution of outliers for the case $p=2$, with $c_1 = 0.6$, $c_2 = 1000$, $N = 10000$, $\xi = 0.1$, and $k = 10$. Data points belonging to the main body are plotted as black circles, whereas outliers are represented by red crosses.
• Figure 2: An illustration of the outlier generation methods for the case $p = 2$, with $c_1 = 0.6$, $c_2 = 1000$, $N = 100$, $\xi = 0.03$, and $k = 10$. The left panel illustrates unclustered outliers, while the right panel illustrates clustered outliers. Data points belonging to the main body are plotted as black circles, whereas outliers are represented by red crosses.
• Figure 3: Plots of the RMSE values of each estimator for different data dimensions $p$, with lines connecting the plotted points. The horizontal axis represents the dimension $p$$( = 2, \ldots, 55 )$, and the vertical axis represents the RMSE. The other settings are $N = 100$, $\xi = 0.03$, and $k = 10$. Figure (a, c, e) show the results for unclustered outliers, while Figure (b, d, f) correspond to clustered outliers. In Figure (a, c, e), the curves representing TME, LNSMI, SSCM, and the proposed method nearly overlap. Furthermore, in Figure (d, f), the curves representing SSCM and the proposed method nearly overlap.
• Figure 4: Plots of the RMSE values of each estimator for different values of the shrinkage coefficient $\alpha$ in the case $p = 25$, with lines connecting the plotted points. The horizontal axis represents the shrinkage coefficient $\alpha$, and the vertical axis represents the RMSE. The other settings are $N = 100$, $\xi = 0.03$, and $k = 10$. Figure (a, c, e) show the results for unclustered outliers, while Figure (b, d, f) correspond to clustered outliers.
• Figure 5: Plots of the RMSE values of each estimator for different values of the shrinkage coefficient $\alpha$ in the case $p = 50$, with lines connecting the plotted points. The horizontal axis represents the shrinkage coefficient $\alpha$, and the vertical axis represents the RMSE. The other settings are $N = 100$, $\xi = 0.03$, and $k = 10$. Figure (a, c, e) show the results for unclustered outliers, while Figure (b, d, f) correspond to clustered outliers.

Theorems & Definitions (24)

• Remark 1
• Remark 2
• Theorem 3.1
• Remark 3
• Corollary 3.1
• Lemma 3.1
• Lemma 4.1
• Lemma 4.2
• Remark 4
• Remark 5