Testing Most Influential Sets

TL;DR

This paper tackles the problem that small data subsets can disproportionately alter model conclusions by providing a principled statistical framework to assess maximal influence. Focusing on linear regression, it derives exact influence formulas and identifies two extreme value regimes: constant-size sets converge to a Fréchet distribution, while relative-size sets converge to a Gumbel distribution, enabling hypothesis tests for excessive influence. The authors propose efficient computation via a closed-form influence expression and an adaptive greedy search, with tail-parameter estimation and block-maxima MLE to fit the appropriate extreme value model. Empirical validation across economics, biology (sparrow morphology), and ML benchmarks demonstrates the method’s ability to distinguish genuine problematic influence from natural sampling variation, supporting more robust and transparent inference in practice.

Abstract

Small subsets of data with disproportionate influence on model outcomes can have dramatic impacts on conclusions, with a few data points sometimes overturning key findings. While recent work has developed methods to identify these most influential sets, no formal theory exists to determine when their influence reflects genuine problems rather than natural sampling variation. We address this gap by developing a principled framework for assessing the statistical significance of most influential sets. Our theoretical results characterize the extreme value distributions of maximal influence and enable rigorous hypothesis tests for excessive influence, replacing current ad-hoc sensitivity checks. We demonstrate the practical value of our approach through applications across economics, biology, and machine learning benchmarks.

Figures (4)

• Figure 1: Illustration of our methodology on a simple linear regression with a moderately influential observation. Panel A depicts observations, estimated regression lines with and without the influential point, and conditional significance regions at the 10, 5, and 1% levels (dotted lines). Panel B shows the extreme value analysis: a histogram of block maxima, fitted Gumbel distribution with (solid) and without (dashed) bias correction, and the resulting $p$-value for the observation of interest.
• Figure A1: Visualization of \ref{['tab:gev_params']}. Most notable is the regime change between thin and polynomial tails. While for the Frechét cases the average MLE is statistically insignificantly different form another, they are all individually different from the Gumbel case. While the finite sample $D^{-1}_{-{\mathbb{S}}}$ has thicker than Normal tails, acting as a regularizer on $\Delta^{\max}$ and thereby resulting in slight underestimation of the theoretically suggested $\xi^{-1}=0.20$.
• Figure A2: Convergence of cases to the limiting distribution, visualizing \ref{['tab:full_shape_simulation']}. Solid lines are estimated means from $1000$ repetitions while the shaded area is $\pm 1SD$. The Normal--Normal case as well as the $t(5)$--$t(5)$ case converge quickly to the theoretical $\xi^{-1}=0$ and $\xi^{-1}=0.2$ respectively. The convergence rate of the mixed cases converge slower yet reasonably fast. In particular, the $t(5)$--Normal case is visibly requiring more observations to stabilize $D^{-1}_{\mathbb{S}}$. In general this finding supports the relevance and applicability of our limiting results to finite samples.
• Figure A3: Simulation exercise for the performance of the simple MLE based on block maxima, correcting for block size. While the corrected location parameter $\hat{\tilde{a}}$ is close to unbiased, the scale parameter $\hat{b}$ suffers some downward bias using simple block maxima, which is in line with dombry2017mle. However, for practical purposes the block maxima is expected to be fitting reasonably well, as visible in panel (a).

Theorems & Definitions (24)

• Definition : Most Influential Set
• Proposition 1
• proof : Proof sketch
• Theorem 1: EVD for constant-size sets
• proof : Proof sketch
• Corollary 1
• Theorem 2: EVD for relative-size sets
• proof : Proof sketch
• Proposition 2
• proof