Data Valuation by Fusing Global and Local Statistical Information

TL;DR

This paper addresses the computational bottlenecks and distribution-ignorance in Shapley-value-based data valuation by uncovering global and local value-distribution patterns and leveraging them through regularization. It introduces GLOC, a distribution-aware refinement of AME that uses a Gaussian-prior (global) and neighborhood-consistency (local) regularizer to improve Shapley-value estimation and extend to dynamic data valuation. The authors propose IncGLOC and DecGLOC to infer updated data values under incremental and decremental changes without re-estimating Shapley values, significantly boosting efficiency. Extensive experiments across twelve datasets demonstrate improved accuracy in Shapley estimation, improved performance in value-based data edits and mislabeled data detection, and substantial computational gains, highlighting the practical impact for data-centric ML and data markets.

Abstract

Data valuation has garnered increasing attention in recent years, given the critical role of high-quality data in various applications. Among diverse data valuation approaches, Shapley value-based methods are predominant due to their strong theoretical grounding. However, the exact computation of Shapley values is often computationally prohibitive, prompting the development of numerous approximation techniques. Despite notable advancements, existing methods generally neglect the incorporation of value distribution information and fail to account for dynamic data conditions, thereby compromising their performance and application potential. In this paper, we highlight the crucial role of both global and local statistical properties of value distributions in the context of data valuation for machine learning. First, we conduct a comprehensive analysis of these distributions across various simulated and real-world datasets, uncovering valuable insights and key patterns. Second, we propose an enhanced data valuation method that fuses the explored distribution characteristics into two regularization terms to refine Shapley value estimation. The proposed regularizers can be seamlessly incorporated into various existing data valuation methods. Third, we introduce a novel approach for dynamic data valuation that infers updated data values without recomputing Shapley values, thereby significantly improving computational efficiency. Extensive experiments have been conducted across a range of tasks, including Shapley value estimation, value-based data addition and removal, mislabeled data detection, and dynamic data valuation. The results showcase the consistent effectiveness and efficiency of our proposed methodologies, affirming the significant potential of global and local value distributions in data valuation.

Figures (9)

• Figure 1: Distributions of data values after min-max normalization for CIFAR10-embeddings (a) and Random (b). The KStest is conducted under the null hypothesis that the data values are Gaussian distributed.The average relative difference after min-max normalization between the value of a sample and the values of its neighbors within the same (c) and different (d) categories. Random dataset defined in Eq. (\ref{['2Gaussion']}) is utilized. The blue and orange curves correspond to samples from the positive (+1) and negative (-1) classes, respectively. $\epsilon$ denotes the neighborhood range.
• Figure 2: Illustration of the global distributions of data values for four additional datasets: BBC, 2Dplanes, Fried, and MiniBooNE. The results of the KStest hypothesis test justel1997multivariate, presented below the figures, indicate that the global value distribution exhibits a closer fit to a Gaussian distribution than to a Laplace distribution.
• Figure 3: Variation in value distribution after adding (a) and removing partial data points (b) from the Random and Electricity datasets.
• Figure 4: Comparison of Shapley value estimation. Ratios of MSEs between AME and GLOC (simplified to the simplest integer ratio) are reported. The MSEs for GLOC are consistently smaller than those for AME, highlighting its superiority in Shapley value estimation.
• Figure 5: Variation in accuracy across different ratios of removed instances. Data points with the highest values are removed first. GLOC exhibits the lowest accuracy, confirming its effectiveness in identifying influential data points.