Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Naive Bayes Classifiers over Missing Data: Decision and Poisoning

Song Bian, Xiating Ouyang, Zhiwei Fan, Paraschos Koutris

TL;DR

The paper addresses certifiable robustness of Naive Bayes classifiers when training data contains missing values, formalizing the Decision Problem and the Data Poisoning Problem. It delivers a polynomial-time decision algorithm with time $O(md+nd)$ and shows that single-point poisoning is solvable in $O(nmd)$ while multi-point poisoning is NP-complete for datasets with at least three features; a practical heuristic is also proposed. Empirical evaluation on ten real-world Kaggle datasets demonstrates substantial speedups over baselines and confirms the method’s effectiveness and robustness to varying missingness. The work provides actionable guarantees that can reduce the need for costly data cleaning and informs security considerations for ML systems operating on dirty data.

Abstract

We study the certifiable robustness of ML classifiers on dirty datasets that could contain missing values. A test point is certifiably robust for an ML classifier if the classifier returns the same prediction for that test point, regardless of which cleaned version (among exponentially many) of the dirty dataset the classifier is trained on. In this paper, we show theoretically that for Naive Bayes Classifiers (NBC) over dirty datasets with missing values: (i) there exists an efficient polynomial time algorithm to decide whether multiple input test points are all certifiably robust over a dirty dataset; and (ii) the data poisoning attack, which aims to make all input test points certifiably non-robust by inserting missing cells to the clean dataset, is in polynomial time for single test points but NP-complete for multiple test points. Extensive experiments demonstrate that our algorithms are efficient and outperform existing baselines.

Naive Bayes Classifiers over Missing Data: Decision and Poisoning

TL;DR

The paper addresses certifiable robustness of Naive Bayes classifiers when training data contains missing values, formalizing the Decision Problem and the Data Poisoning Problem. It delivers a polynomial-time decision algorithm with time and shows that single-point poisoning is solvable in while multi-point poisoning is NP-complete for datasets with at least three features; a practical heuristic is also proposed. Empirical evaluation on ten real-world Kaggle datasets demonstrates substantial speedups over baselines and confirms the method’s effectiveness and robustness to varying missingness. The work provides actionable guarantees that can reduce the need for costly data cleaning and informs security considerations for ML systems operating on dirty data.

Abstract

We study the certifiable robustness of ML classifiers on dirty datasets that could contain missing values. A test point is certifiably robust for an ML classifier if the classifier returns the same prediction for that test point, regardless of which cleaned version (among exponentially many) of the dirty dataset the classifier is trained on. In this paper, we show theoretically that for Naive Bayes Classifiers (NBC) over dirty datasets with missing values: (i) there exists an efficient polynomial time algorithm to decide whether multiple input test points are all certifiably robust over a dirty dataset; and (ii) the data poisoning attack, which aims to make all input test points certifiably non-robust by inserting missing cells to the clean dataset, is in polynomial time for single test points but NP-complete for multiple test points. Extensive experiments demonstrate that our algorithms are efficient and outperform existing baselines.
Paper Structure (30 sections, 5 theorems, 51 equations, 10 figures, 1 table, 4 algorithms)

This paper contains 30 sections, 5 theorems, 51 equations, 10 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Decision Algorithms
  4. Data Poisoning Algorithms
  5. A Single Test Point
  6. Multiple Test Points
  7. Experiments
  8. Experimental Setup
  9. Algorithms Evaluated
  10. Decision Problem.
  11. Data Poisoning Problem.
  12. Evaluation Results
  13. Decision Problems.
  14. Data Poisoning Problem.
  15. Related Work
  16. ...and 15 more sections

Key Result

Lemma 3.1

A test point $\bm{t}$ is certifiably robust for NBC over $\mathcal{D}^\square$ if and only if there is a label $l$ such that for any label $l' \neq l$, $S_{{\mathcal{D}^\square}}^{\downarrow}({l} \mid {\bm{t}}) > S_{{\mathcal{D}^\square}}^{\uparrow}({l'} \mid {\bm{t}}).$

Figures (10)

  • Figure 1: Example of an incomplete dataset, the possible worlds induced by the incomplete dataset, and the certifiably robust/non-robust predictions for data points with missing values over the possible worlds.
  • Figure 2: Decision - Running Time vs Missing Rates on Different Datasets
  • Figure 3: Decision - Running Time vs Number of Test Points on Different Datasets
  • Figure 4: Single Point Data Poisoning---Running Time and Poisoning Rate on Different Datasets
  • Figure 5: The incomplete data $\mathcal{D}^\square$ has (among others) two possible worlds $\mathcal{D}$ and $\mathcal{D}^*$ shown in (b) and (c) respectively.
  • ...and 5 more figures

Theorems & Definitions (16)

  • Example 1.1
  • Lemma 3.1
  • Definition 4.1: AlterPrediction
  • Lemma 4.2
  • Lemma 4.3
  • Theorem 4.4
  • proof
  • Example 1.1
  • proof
  • Lemma 2.1
  • ...and 6 more