Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Thresholding Data Shapley for Data Cleansing Using Multi-Armed Bandits

Hiroyuki Namba, Shota Horiguchi, Masaki Hamamoto, Masashi Egi

TL;DR

An iterative method to fast identify a subset of instances with low data Shapley values by using the thresholding bandit algorithm is proposed and provides a theoretical guarantee that the proposed method can accurately select harmful instances if a sufficiently large number of iterations is conducted.

Abstract

Data cleansing aims to improve model performance by removing a set of harmful instances from the training dataset. Data Shapley is a common theoretically guaranteed method to evaluate the contribution of each instance to model performance; however, it requires training on all subsets of the training data, which is computationally expensive. In this paper, we propose an iterativemethod to fast identify a subset of instances with low data Shapley values by using the thresholding bandit algorithm. We provide a theoretical guarantee that the proposed method can accurately select harmful instances if a sufficiently large number of iterations is conducted. Empirical evaluation using various models and datasets demonstrated that the proposed method efficiently improved the computational speed while maintaining the model performance.

Thresholding Data Shapley for Data Cleansing Using Multi-Armed Bandits

TL;DR

An iterative method to fast identify a subset of instances with low data Shapley values by using the thresholding bandit algorithm is proposed and provides a theoretical guarantee that the proposed method can accurately select harmful instances if a sufficiently large number of iterations is conducted.

Abstract

Data cleansing aims to improve model performance by removing a set of harmful instances from the training dataset. Data Shapley is a common theoretically guaranteed method to evaluate the contribution of each instance to model performance; however, it requires training on all subsets of the training data, which is computationally expensive. In this paper, we propose an iterativemethod to fast identify a subset of instances with low data Shapley values by using the thresholding bandit algorithm. We provide a theoretical guarantee that the proposed method can accurately select harmful instances if a sufficiently large number of iterations is conducted. Empirical evaluation using various models and datasets demonstrated that the proposed method efficiently improved the computational speed while maintaining the model performance.
Paper Structure (29 sections, 3 theorems, 21 equations, 2 figures, 7 tables, 1 algorithm)

This paper contains 29 sections, 3 theorems, 21 equations, 2 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Data Shapley
  4. Thresholding Bandits
  5. Proposed Method
  6. Formulation as Thresholding Multi-armed Bandit Problem
  7. Basic algorithm
  8. Further enhancements
  9. Theoretical Results
  10. Upper bound of Failure Probability
  11. Upper Bound of Width
  12. Experiments
  13. Settings
  14. Evaluation using Various Models
  15. Prediction performance
  16. ...and 14 more sections

Key Result

Theorem 1

Assume that $T\geq 256R^2H\log(N(1+\log T))$ and the reward distribution $P_n$ is $R$-sub-Gaussian for all $n\in\mathcal{N}$. For the APT algorithm, it holds that

Figures (2)

  • Figure 1: MAEs of DTree when varying the number of instances to be removed on the Abalone dataset.
  • Figure 2: MAEs when varying the number of instances to be removed on the Abalone dataset (test set).

Theorems & Definitions (5)

  • Theorem 1
  • Theorem 2
  • proof : Proof of Theorem \ref{['the:2']}
  • Proposition 1
  • proof : Proof of Proposition \ref{['the:wbound']}