Strategyproof Learning with Advice
Eric Balkanski, Cherlin Zhu
TL;DR
This work introduces strategyproof learning with advice, integrating an advisory function $ ilde{f}$ into mechanism design to improve performance when the advice is accurate and maintain robustness when it is not. It presents the Constant Fit with Advice (CFA) mechanism for constant function classes, proving it is strategyproof and achieves a tight consistency-robustness tradeoff of $1+ ext{γ}$ and $1+4/ ext{γ}$, with extensions to homogeneous linear functions via the Linear Fit with Advice (LFA) mechanism. The authors also establish strong impossibility results for classification with advice when considering more than two labelings, while showing that the two-labelings case admits efficient, near-optimal strategyproof randomized/deterministic mechanisms. A learning-theoretic extension demonstrates that, under standard PAC/Rademacher assumptions, empirical-risk-based strategies with advice generalize to distributional settings, preserving the proposed guarantees with high probability. Overall, the paper clarifies when side advice can meaningfully improve strategyproof learning and highlights fundamental limits in strategic classification under advisory information.
Abstract
An important challenge in robust machine learning is when training data is provided by strategic sources who may intentionally report erroneous data for their own benefit. A line of work at the intersection of machine learning and mechanism design aims to deter strategic agents from reporting erroneous training data by designing learning algorithms that are strategyproof. Strategyproofness is a strong and desirable property, but it comes at a cost in the approximation ratio of even simple risk minimization problems. In this paper, we study strategyproof regression and classification problems in a model with advice. This model is part of a recent line on mechanism design with advice where the goal is to achieve both an improved approximation ratio when the advice is correct (consistency) and a bounded approximation ratio when the advice is incorrect (robustness). We provide the first non-trivial consistency-robustness tradeoffs for strategyproof regression and classification, which hold for simple yet interesting classes of functions. For classes of constant functions, we give a deterministic and strategyproof mechanism that is, for any $γ\in (0, 2]$, $1+γ$ consistent and $1 + 4/γ$ robust and provide a lower bound that shows that this tradeoff is optimal. We extend this mechanism and its guarantees to homogeneous linear regression over $\mathbb{R}$. In the binary classification problem of selecting from three or more labelings, we present strong impossibility results for both deterministic and randomized mechanism. Finally, we provide deterministic and randomized mechanisms for selecting from two labelings.