Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strategic Littlestone Dimension: Improved Bounds on Online Strategic Classification

Saba Ahmadi, Kunhe Yang, Hanrui Zhang

TL;DR

This work studies online binary classification with strategic agents who can manipulate their observable features via a manipulation graph, so the learner observes post-manipulation data. It introduces the Strategic Littlestone Dimension, $\mathsf{SLdim}(\mathcal{H},G)$, and proves that in the realizable setting the instance-optimal mistake bound equals $\mathsf{SLdim}(\mathcal{H},G)$, implemented by the Strategic Standard Optimal Algorithm. In the agnostic setting, it yields improved regret through a refined agnostic-to-realizable reduction using a finite expert set and biased voting. It further extends to learning when the manipulation graph is unknown by considering a graph class $\mathcal{G}$ and deriving regret bounds for both realizable and agnostic graph-class scenarios, including when agents select different graphs. These results unify and sharpen prior bounds, revealing new directions for randomized learners and robustness to graph uncertainty.

Abstract

We study the problem of online binary classification in settings where strategic agents can modify their observable features to receive a positive classification. We model the set of feasible manipulations by a directed graph over the feature space, and assume the learner only observes the manipulated features instead of the original ones. We introduce the Strategic Littlestone Dimension, a new combinatorial measure that captures the joint complexity of the hypothesis class and the manipulation graph. We demonstrate that it characterizes the instance-optimal mistake bounds for deterministic learning algorithms in the realizable setting. We also achieve improved regret in the agnostic setting by a refined agnostic-to-realizable reduction that accounts for the additional challenge of not observing agents' original features. Finally, we relax the assumption that the learner knows the manipulation graph, instead assuming their knowledge is captured by a family of graphs. We derive regret bounds in both the realizable setting where all agents manipulate according to the same graph within the graph family, and the agnostic setting where the manipulation graphs are chosen adversarially and not consistently modeled by a single graph in the family.

Strategic Littlestone Dimension: Improved Bounds on Online Strategic Classification

TL;DR

This work studies online binary classification with strategic agents who can manipulate their observable features via a manipulation graph, so the learner observes post-manipulation data. It introduces the Strategic Littlestone Dimension, , and proves that in the realizable setting the instance-optimal mistake bound equals , implemented by the Strategic Standard Optimal Algorithm. In the agnostic setting, it yields improved regret through a refined agnostic-to-realizable reduction using a finite expert set and biased voting. It further extends to learning when the manipulation graph is unknown by considering a graph class and deriving regret bounds for both realizable and agnostic graph-class scenarios, including when agents select different graphs. These results unify and sharpen prior bounds, revealing new directions for randomized learners and robustness to graph uncertainty.

Abstract

We study the problem of online binary classification in settings where strategic agents can modify their observable features to receive a positive classification. We model the set of feasible manipulations by a directed graph over the feature space, and assume the learner only observes the manipulated features instead of the original ones. We introduce the Strategic Littlestone Dimension, a new combinatorial measure that captures the joint complexity of the hypothesis class and the manipulation graph. We demonstrate that it characterizes the instance-optimal mistake bounds for deterministic learning algorithms in the realizable setting. We also achieve improved regret in the agnostic setting by a refined agnostic-to-realizable reduction that accounts for the additional challenge of not observing agents' original features. Finally, we relax the assumption that the learner knows the manipulation graph, instead assuming their knowledge is captured by a family of graphs. We derive regret bounds in both the realizable setting where all agents manipulate according to the same graph within the graph family, and the agnostic setting where the manipulation graphs are chosen adversarially and not consistently modeled by a single graph in the family.
Paper Structure (42 sections, 11 theorems, 31 equations, 1 figure, 9 algorithms)

This paper contains 42 sections, 11 theorems, 31 equations, 1 figure, 9 algorithms.

Table of Contents

  1. Introduction
  2. Our Results and Techniques
  3. An instance-optimal regret bound through the strategic Littlestone dimension.
  4. An improved regret bound for the agnostic case.
  5. Learning with unknown manipulation graphs.
  6. Further Related Work
  7. Model and Preliminaries
  8. Strategic classification.
  9. Manipulation rule.
  10. Online learning.
  11. Classical Littlestone Dimension
  12. The Strategic Littlestone Dimension
  13. Agnostic Setting
  14. Unknown Manipulation Graph
  15. Realizable graph classes
  16. ...and 27 more sections

Key Result

Proposition 2.3

Let $\mathcal{M}(\mathcal{H})$ be the optimal mistake in the classical online learning setting, then $\mathcal{M}(\mathcal{H})=\mathsf{Ldim}(\mathcal{H})$.

Figures (1)

  • Figure 1: A Strategic Littlestone Tree with depth $2$. False negative edges are marked red, whereas false positive edges are marked blue. The highlighted path $x_1' \xrightarrow{(v_1,y_1)} x_2' \xrightarrow{(v_2,y_2)}x_{3}'$ is an example root-to-leaf path. In this path, the first observation $(v_1,y_1)$ is a false positive, which satisfies $v_1\in N_G^+[x_1']$ and $y_1=-1$; the second observation $(v_2,y_2)$ is a false negative, which satisfies $v_2=x_2'$ and $y_2=+1$.

Theorems & Definitions (27)

  • Definition 2.1: $\mathcal{H}$-Shattered Littlestone Tree
  • Definition 2.2: Littlestone Dimension
  • Proposition 2.3: Optimal Mistake Bound littlestone1988learning
  • Definition 3.1: $\mathcal{H}$-Shattered Strategic Littlestone Tree
  • Definition 3.2: Strategic Littlestone Dimension
  • Theorem 3.3: Minmax optimal mistake for strategic classification
  • Theorem 3.4: Lower bound part of \ref{['thm:main-sldim']}
  • Theorem 3.5: Upper bound part of \ref{['thm:main-sldim']}
  • Remark 3.6: Comparison with previous results
  • Theorem 4.1
  • ...and 17 more