Verifying Classification with Limited Disclosure
Siddharth Bhandari, Liren Shan
TL;DR
The paper addresses verifying classifier correctness in a privacy-preserving multi-party e-discovery setting. It introduces the Leave-One-Out dimension to quantify how many nonresponsive documents must be disclosed to certify a classifier's correctness, and shows that for realizable problems this bound is tight up to a constant; for linear classifiers with margin, there is a sharp trichotomy depending on γ relative to 1/3. The authors extend the framework to nonrealizable scenarios using a Robust-Leave-One-Out dimension and develop error-tolerant variants to handle mislabeling, all while keeping disclosure of nonresponsive documents minimal. This yields privacy-preserving, verifiable classification protocols with practical implications for sensitive document review and potential applications in coding theory and combinatorial geometry.
Abstract
We consider the multi-party classification problem introduced by Dong, Hartline, and Vijayaraghavan (2022) motivated by electronic discovery. In this problem, our goal is to design a protocol that guarantees the requesting party receives nearly all responsive documents while minimizing the disclosure of nonresponsive documents. We develop verification protocols that certify the correctness of a classifier by disclosing a few nonresponsive documents. We introduce a combinatorial notion called the Leave-One-Out dimension of a family of classifiers and show that the number of nonresponsive documents disclosed by our protocol is at most this dimension in the realizable setting, where a perfect classifier exists in this family. For linear classifiers with a margin, we characterize the trade-off between the margin and the number of nonresponsive documents that must be disclosed for verification. Specifically, we establish a trichotomy in this requirement: for $d$ dimensional instances, when the margin exceeds $1/3$, verification can be achieved by revealing only $O(1)$ nonresponsive documents; when the margin is exactly $1/3$, in the worst case, at least $Ω(d)$ nonresponsive documents must be disclosed; when the margin is smaller than $1/3$, verification requires $Ω(e^d)$ nonresponsive documents. We believe this result is of independent interest with applications to coding theory and combinatorial geometry. We further extend our protocols to the nonrealizable setting defining an analogous combinatorial quantity robust Leave-One-Out dimension, and to scenarios where the protocol is tolerant to misclassification errors by Alice.