Strategic Classification with Non-Linear Classifiers

TL;DR

This work advances strategic classification by analyzing non-linear classifiers, showing that strategic user behavior can both inflate and deflate effective boundary complexity and that universal approximators are no longer universally powerful under strategic manipulation. It develops a bottom-up, geometry-based framework that ties point motions to boundary changes via mappings, curvature bounds, and containment effects, yielding both upper and lower bounds on VC dimensions for broad classes, including piecewise-linear and polytopes. The results reveal approximation gaps and non-universality, with theoretical bounds complemented by experiments that illustrate expressivity shifts and strategic-accuracy limitations. The findings have practical implications for model-class selection, loss design, and optimization in settings where users actively react to deployed classifiers.

Abstract

In strategic classification, the standard supervised learning setting is extended to support the notion of strategic user behavior in the form of costly feature manipulations made in response to a classifier. While standard learning supports a broad range of model classes, the study of strategic classification has, so far, been dedicated mostly to linear classifiers. This work aims to expand the horizon by exploring how strategic behavior manifests under non-linear classifiers and what this implies for learning. We take a bottom-up approach showing how non-linearity affects decision boundary points, classifier expressivity, and model class complexity. Our results show how, unlike the linear case, strategic behavior may either increase or decrease effective class complexity, and that the complexity decrease may be arbitrarily large. Another key finding is that universal approximators (e.g., neural nets) are no longer universal once the environment is strategic. We demonstrate empirically how this can create performance gaps even on an unrestricted model class.

Figures (11)

• Figure 1: (Left) Original non-linear classifier $h$. (Middle) Strategic response to original classifier. (Right) Resulting effective classifier$h_\Delta$ (different from $h$) after accounting for strategic responses.
• Figure 2: Types of point mappings. From left to right: one-to-one (case #1), direct wipeout (case #2), indirect wipeout (case #2), expansion (case #3), and collision (case #4). Black lines show original $h$, dashed gray lines indicate $h_\Delta$, gold arrows show ${z} \mapsto \nabla_h({z})$, and dotted circles show $S_\alpha(x)$
• Figure 3: Types of impossible effective classifiers. From left to right: small positive region, narrow positive strip, large positive curvature, piecewise (hyper)linear convex towards the positive region.
• Figure 4: (Left) Example of increasing VC. (Right) Example of a dataset with limited strategic accuracy. Any $h_\Delta$ which correctly classifies the gold-rimmed point must err on a negative point.
• Figure 5: (Left) Expressivity. For random polynomial classifiers of degree $k$, results show the smallest degree $k'$ that captures the effective decision boundary. (Center) An instance showing positive curvature decreasing (A), negative curvature increasing (B), and wipeout (C). (Right) Approximation. As data becomes more entangled (low separation), strategic approximation degrades.

Theorems & Definitions (34)

• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Theorem 1
• Corollary 1
• Theorem 2
• Theorem 3
• Theorem 4
• Corollary 2