A Unified Characterization of Private Learnability via Graph Theory
Noga Alon, Shay Moran, Hilla Schefler, Amir Yehudayoff
TL;DR
This work introduces a unified graph-theoretic framework that links private learnability to the structure of contradiction graphs, revealing a precise dichotomy: finite fractional clique dimension $\mathrm{CD}^*(\mathcal{H})$ characterizes pure DP learnability, while finite clique dimension $\mathrm{CD}(\mathcal{H})$ characterizes approximate DP learnability. A central theme is the strong duality between fractional cliques and fractional colorings, extended to possibly infinite graphs, which underpins connections to representation dimension and Littlestone dimension. The authors develop two core dichotomies, relate graph-theoretic dimensions to classic learning-theoretic measures, and provide a suite of proofs (some via online learning, LP duality, and minimax theorems) that crystallize how private learnability emerges from combinatorial graph structure. The results open new avenues for applying graph theory to learning theory and pose several open questions about tightening the relationships between the dimensions and exploring direct constructions from large cliques or fractional cliques. Overall, the paper offers a rigorous, novel lens for understanding what makes learning task privately tractable.
Abstract
We provide a unified framework for characterizing pure and approximate differentially private (DP) learnability. The framework uses the language of graph theory: for a concept class $\mathcal{H}$, we define the contradiction graph $G$ of $\mathcal{H}$. Its vertices are realizable datasets, and two datasets $S,S'$ are connected by an edge if they contradict each other (i.e., there is a point $x$ that is labeled differently in $S$ and $S'$). Our main finding is that the combinatorial structure of $G$ is deeply related to learning $\mathcal{H}$ under DP. Learning $\mathcal{H}$ under pure DP is captured by the fractional clique number of $G$. Learning $\mathcal{H}$ under approximate DP is captured by the clique number of $G$. Consequently, we identify graph-theoretic dimensions that characterize DP learnability: the clique dimension and fractional clique dimension. Along the way, we reveal properties of the contradiction graph which may be of independent interest. We also suggest several open questions and directions for future research.