Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One-Shot Strategic Classification Under Unknown Costs

Elan Rosenfeld, Nir Rosenfeld

TL;DR

Focusing on uncertainty in the users' cost function, it is proved that for a broad class of costs, even a small mis-estimation of the true cost can entail trivial accuracy in the worst case, and frames the task as a minimax problem, aiming to minimize worst-case risk over an uncertainty set of costs.

Abstract

The goal of strategic classification is to learn decision rules which are robust to strategic input manipulation. Earlier works assume that these responses are known; while some recent works handle unknown responses, they exclusively study online settings with repeated model deployments. But there are many domains$\unicode{x2014}$particularly in public policy, a common motivating use case$\unicode{x2014}$where multiple deployments are infeasible, or where even one bad round is unacceptable. To address this gap, we initiate the formal study of one-shot strategic classification under unknown responses, which requires committing to a single classifier once. Focusing on uncertainty in the users' cost function, we begin by proving that for a broad class of costs, even a small mis-estimation of the true cost can entail trivial accuracy in the worst case. In light of this, we frame the task as a minimax problem, aiming to minimize worst-case risk over an uncertainty set of costs. We design efficient algorithms for both the full-batch and stochastic settings, which we prove converge (offline) to the minimax solution at the rate of $\tilde{\mathcal{O}}(T^{-\frac{1}{2}})$. Our analysis reveals important structure stemming from strategic responses, particularly the value of dual norm regularization with respect to the cost function.

One-Shot Strategic Classification Under Unknown Costs

TL;DR

Focusing on uncertainty in the users' cost function, it is proved that for a broad class of costs, even a small mis-estimation of the true cost can entail trivial accuracy in the worst case, and frames the task as a minimax problem, aiming to minimize worst-case risk over an uncertainty set of costs.

Abstract

The goal of strategic classification is to learn decision rules which are robust to strategic input manipulation. Earlier works assume that these responses are known; while some recent works handle unknown responses, they exclusively study online settings with repeated model deployments. But there are many domainsparticularly in public policy, a common motivating use casewhere multiple deployments are infeasible, or where even one bad round is unacceptable. To address this gap, we initiate the formal study of one-shot strategic classification under unknown responses, which requires committing to a single classifier once. Focusing on uncertainty in the users' cost function, we begin by proving that for a broad class of costs, even a small mis-estimation of the true cost can entail trivial accuracy in the worst case. In light of this, we frame the task as a minimax problem, aiming to minimize worst-case risk over an uncertainty set of costs. We design efficient algorithms for both the full-batch and stochastic settings, which we prove converge (offline) to the minimax solution at the rate of . Our analysis reveals important structure stemming from strategic responses, particularly the value of dual norm regularization with respect to the cost function.
Paper Structure (38 sections, 25 theorems, 59 equations, 1 figure, 3 algorithms)

This paper contains 38 sections, 25 theorems, 59 equations, 1 figure, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Strategic classification.
  4. (Initially) unknown user responses.
  5. Preliminaries
  6. Notation.
  7. Form of the cost function.
  8. Parameterizing the space of costs.
  9. Encoding uncertainty.
  10. Strategic Learning Under a Single, Known Cost
  11. Cost-aware strategic hinge.
  12. Risk and generalization.
  13. The Perils of Using a Wrong Cost Function
  14. Hardness results.
  15. Intentionally conservative guesses can still fail.
  16. ...and 23 more sections

Key Result

Lemma 2.3

Fix $\beta$ and let $c(x, x') = \phi(\| x-x' \|_{\Sigma})$. The maximum possible change to a user's score due to strategic behavior is $u_{*} \| \beta \|_{*,\Sigma}$.

Figures (1)

  • Figure 1: Visualizing the excess 0-1 risk from \ref{['thm:cost-error-gauss-lower-bound']}. Left: A toy illustration of the excess risk as a function of $\| \mu_0 \|$ and $\epsilon$ and where they lie on the Gaussian CDF. Right: Excess risk curves as a function of the dimension $d$, where $\mu_0 = 1/\sqrt{d}\; \mathbf{1}$ and $\sigma^2 = 1/d$. Color indicates the function which generates the spectrum of $\Sigma$, where $\lambda_k$ is the $k$-largest eigenvalue in the series. Line style indicates the error $\varepsilon$ in estimating the eigenvalue in each dimension---precisely, we define $\hat{\Sigma} = (1 - \varepsilon) \Sigma$. Even with all eigenvalues estimated to error $1-e^{-10}$, excess risk grows rapidly with $d$ towards $1/2$.

Theorems & Definitions (38)

  • Definition 2.1
  • Definition 2.2
  • Lemma 2.3
  • Lemma 2.4
  • Theorem 2.5
  • Theorem 3.1
  • Theorem 3.2
  • Lemma 4.1
  • Theorem 4.2
  • Proposition 4.3
  • ...and 28 more