Unveil Sources of Uncertainty: Feature Contribution to Conformal Prediction Intervals

TL;DR

This work tackles the challenge of attributing predictive uncertainty in ML to input features. It couples conformal prediction with cooperative game theory to create CP-based value functions that capture interval width and bounds, and then distributes these uncertainty contributions using Shapley and proportional Shapley allocations. A Monte Carlo approximation with unbiased, consistent, and asymptotically normal guarantees enables scalable computation, including an importance-sampling variant for efficiency. Experiments on synthetic benchmarks and real-world datasets reveal that CP-based uncertainty attributions can diverge from moment-based rankings, offering a richer and more reliable interpretive tool for high-stakes decisions where predictive uncertainty matters.

Abstract

Cooperative game theory methods, notably Shapley values, have significantly enhanced machine learning (ML) interpretability. However, existing explainable AI (XAI) frameworks mainly attribute average model predictions, overlooking predictive uncertainty. This work addresses that gap by proposing a novel, model-agnostic uncertainty attribution (UA) method grounded in conformal prediction (CP). By defining cooperative games where CP interval properties-such as width and bounds-serve as value functions, we systematically attribute predictive uncertainty to input features. Extending beyond the traditional Shapley values, we use the richer class of Harsanyi allocations, and in particular the proportional Shapley values, which distribute attribution proportionally to feature importance. We propose a Monte Carlo approximation method with robust statistical guarantees to address computational feasibility, significantly improving runtime efficiency. Our comprehensive experiments on synthetic benchmarks and real-world datasets demonstrate the practical utility and interpretative depth of our approach. By combining cooperative game theory and conformal prediction, we offer a rigorous, flexible toolkit for understanding and communicating predictive uncertainty in high-stakes ML applications.

Figures (12)

• Figure 1: Empirical convergence for the four most important features (A), number of trained models and worst case scenario $m \times d$ (B), and runtime relative to exact computations (C) for the empirical study of the Monte Carlo approximation scheme
• Figure 2: Feature attribution for the modified Friedman example. The bars mark the 90% intervals.
• Figure 3: LACP width-based vs. conditional mean-based importance rankings for RF and LGB models on the concrete dataset.
• Figure 4: Rank frequency of all the features (A) and top 5 most important feature (B) over the test data. CQR upper bound-based importance rankings for Q-LR and Q-RF models on the facebook dataset are used.
• Figure 5: Empirical convergence of Monte Carlo Shapley value estimates. For each number of permutations (y-axis), a black dot indicates the mean Shapley value over 150 replications, and horizontal bars indicate $\pm1$ standard deviation. The orange vertical line marks the Shapley value computed with the exact estimation procedure.

Theorems & Definitions (15)

• Proposition 3.1
• Theorem 1: Statistical properties of Algorithm \ref{['alg:WeberPermut']}
• Theorem 2: Statistical properties of the IS approximation
• Remark : Shapley values and model explanations
• Proposition A.1: Efficiency of the Weber set
• proof : Proof of Proposition \ref{['prop:effWeber']}
• Example : Random order formulation for 3 players
• Proposition A.2: Efficiency of the Harsanyi set
• proof : Proof of Proposition \ref{['prop:effHarsa']}
• Example : Dividend-sharing formulation for 3 players