Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distribution Learnability and Robustness

Shai Ben-David, Alex Bie, Gautam Kamath, Tosca Lechner

TL;DR

It is shown that realizable learnability of a class of distributions implies its robust learnability with respect to only additive corruption, but not against subtractive corruption, and related implications in the context of compression schemes and differentially private learnability.

Abstract

We examine the relationship between learnability and robust (or agnostic) learnability for the problem of distribution learning. We show that, contrary to other learning settings (e.g., PAC learning of function classes), realizable learnability of a class of probability distributions does not imply its agnostic learnability. We go on to examine what type of data corruption can disrupt the learnability of a distribution class and what is such learnability robust against. We show that realizable learnability of a class of distributions implies its robust learnability with respect to only additive corruption, but not against subtractive corruption. We also explore related implications in the context of compression schemes and differentially private learnability.

Distribution Learnability and Robustness

TL;DR

It is shown that realizable learnability of a class of distributions implies its robust learnability with respect to only additive corruption, but not against subtractive corruption, and related implications in the context of compression schemes and differentially private learnability.

Abstract

We examine the relationship between learnability and robust (or agnostic) learnability for the problem of distribution learning. We show that, contrary to other learning settings (e.g., PAC learning of function classes), realizable learnability of a class of probability distributions does not imply its agnostic learnability. We go on to examine what type of data corruption can disrupt the learnability of a distribution class and what is such learnability robust against. We show that realizable learnability of a class of distributions implies its robust learnability with respect to only additive corruption, but not against subtractive corruption. We also explore related implications in the context of compression schemes and differentially private learnability.

Paper Structure

This paper contains 23 sections, 25 theorems, 57 equations.

Table of Contents

  1. Introduction
  2. Definitions of Learnability
  3. Results and Techniques
  4. Adaptive Adversaries.
  5. Known vs unknown level of corruption
  6. Related Work
  7. Learnability Does Not Imply Robust (Agnostic) Learnablity
  8. Proof of Main Result
  9. Proof of Claim \ref{['claim:Q_grealizable']}
  10. Proof of Claim \ref{['claim:Q_grobust']}
  11. Proof of Theorem \ref{['thm:realizable-/->subtractive']}
  12. Learnability Implies Additive Robust Learnability
  13. Subtractive Robust Learnability Implies Robust Learnability
  14. Proof of Theorem \ref{['thm:additive']}
  15. Existence of sample compression schemes
  16. ...and 8 more sections

Key Result

Theorem 1.5

There exists a class that is realizably learnable, but not agnostically/robustly learnable.

Theorems & Definitions (52)

  • Definition 1.1: Learnability
  • Definition 1.2: Robust learnability
  • Definition 1.3: Additively robust learnability
  • Definition 1.4: Subtractive robust learnability
  • Theorem 1.5: Informal
  • Theorem 1.6: Informal
  • Theorem 1.7: Informal
  • Theorem 1.8: Informal
  • Theorem 1.9: Informal
  • Theorem 1.10: Informal
  • ...and 42 more