Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transductive Learning Is Compact

Julian Asilis, Siddartha Devic, Shaddin Dughmi, Vatsal Sharan, Shang-Hua Teng

Abstract

We demonstrate a compactness result holding broadly across supervised learning with a general class of loss functions: Any hypothesis class $H$ is learnable with transductive sample complexity $m$ precisely when all of its finite projections are learnable with sample complexity $m$. We prove that this exact form of compactness holds for realizable and agnostic learning with respect to any proper metric loss function (e.g., any norm on $\mathbb{R}^d$) and any continuous loss on a compact space (e.g., cross-entropy, squared loss). For realizable learning with improper metric losses, we show that exact compactness of sample complexity can fail, and provide matching upper and lower bounds of a factor of 2 on the extent to which such sample complexities can differ. We conjecture that larger gaps are possible for the agnostic case. Furthermore, invoking the equivalence between sample complexities in the PAC and transductive models (up to lower order factors, in the realizable case) permits us to directly port our results to the PAC model, revealing an almost-exact form of compactness holding broadly in PAC learning.

Transductive Learning Is Compact

Abstract

We demonstrate a compactness result holding broadly across supervised learning with a general class of loss functions: Any hypothesis class is learnable with transductive sample complexity precisely when all of its finite projections are learnable with sample complexity . We prove that this exact form of compactness holds for realizable and agnostic learning with respect to any proper metric loss function (e.g., any norm on ) and any continuous loss on a compact space (e.g., cross-entropy, squared loss). For realizable learning with improper metric losses, we show that exact compactness of sample complexity can fail, and provide matching upper and lower bounds of a factor of 2 on the extent to which such sample complexities can differ. We conjecture that larger gaps are possible for the agnostic case. Furthermore, invoking the equivalence between sample complexities in the PAC and transductive models (up to lower order factors, in the realizable case) permits us to directly port our results to the PAC model, revealing an almost-exact form of compactness holding broadly in PAC learning.
Paper Structure (18 sections, 16 theorems, 15 equations, 1 figure)

This paper contains 18 sections, 16 theorems, 15 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Preliminaries
  5. Notation
  6. Transductive Learning
  7. Compactness of Learning
  8. Realizable Learning With Proper Loss Functions
  9. Realizable Learning With Improper Loss Functions
  10. Agnostic Learning
  11. PAC Learning
  12. Conclusion
  13. Proper metric spaces
  14. Distribution-family Learning
  15. Omitted proofs
  16. ...and 3 more sections

Key Result

Theorem 3.3

Let $L$ be a collection of variables, with each variable $\ell \in L$ taking values in a metric space $M_\ell$. Let $R$ be a collection of proper functions, each of which depends upon finitely many variables in $L$ and has codomain $\mathbb R_{\geq 0}$. Then the following conditions are equivalent f

Figures (1)

  • Figure 1: Depiction of variables $L$ and functions $R$ which model transductive learning, for a sequence of unlabeled datapoints $|S| = 3$ such that $\mathcal{H}|_S$ contains the behaviors $(0, 0, 0)$, $(1, 0, 0)$, and $(0, 1, 0)$. Arrows denote functional dependence, i.e., each $r \in R$ depends upon its incident variables.

Theorems & Definitions (40)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Definition 3.1
  • Definition 3.2
  • Theorem 3.3
  • proof
  • Remark 3.5
  • Theorem 3.6
  • proof
  • ...and 30 more