Incentive Aware AI Regulations: A Credal Characterisation

TL;DR

This work proves that a mechanism has perfect market outcome if and only if the set of non-compliant distributions forms a credal set, i.e., a closed, convex set of probability measures.

Abstract

While high-stakes ML applications demand strict regulations, strategic ML providers often evade them to lower development costs. To address this challenge, we cast AI regulation as a mechanism design problem under uncertainty and introduce regulation mechanisms: a framework that maps empirical evidence from models to a license for some market share. The providers can select from a set of licenses, effectively forcing them to bet on their model's ability to fulfil regulation. We aim at regulation mechanisms that achieve perfect market outcome, i.e. (a) drive non-compliant providers to self-exclude, and (b) ensure participation from compliant providers. We prove that a mechanism has perfect market outcome if and only if the set of non-compliant distributions forms a credal set, i.e., a closed, convex set of probability measures. This result connects mechanism design and imprecise probability by establishing a duality between regulation mechanisms and the set of non-compliant distributions. We also demonstrate these mechanisms in practice via experiments on regulating use of spurious features for prediction and fairness. Our framework provides new insights at the intersection of mechanism design and imprecise probability, offering a foundation for development of enforceable AI regulations.

Figures (4)

• Figure 1: An illustration for Theorem \ref{['theorem:obedience-to-credal']} for a classification task with $K=3$ classes. The blue regions represent the set of non-compliant distributions $\mathcal{P}_0$ within the probability simplex. (Left) A non-credal $\mathcal{P}_0$ fails obedience: a provider can construct a compliant mixture (red dot) from two non-compliant models (dark blue dots), thus bypassing the regulation. (Middle) Non-credal $\mathcal{P}_0$ further violates feasibility, as there are compliant distributions that cannot be separated from such a non-credal $\mathcal{P}_0$ by a linear functional (dotted red line), making perfect market outcome impossible. (Right) When $\mathcal{P}_0$ is a credal set, a linear separating hyperplane exists, guaranteeing an implementable $\Pi$.
• Figure 2: From Left-to-Right: Fig (a) demonstrates that a naive regulator with a non-convex $\mathcal{P}_0$ can be gamed by a strategic provider by mixing evidence from bad models. Fig (b) plots $\pi^*$ vs samples $n$ for ERM (non-compliant) and Group-DRO (compliant) agents on Waterbirds. Fig (c) plots Ratio $\pi^*_{\mathrm{DRO}} / \pi^*_{\mathrm{ERM}}$ evaluated on 100 random test samples, separated into easy (majority) and hard (minority, counter-spurious) examples to show Group-DRO agent gets better license due to its performance on hard examples. Fig (d) shows the practical regulations for fairness based when credal set is implicit. All figures are for 30 runs and shaded regions indicate standard error.
• Figure 3: Power vs Type 1 in the Chi-squared test of model parameters/features i.e. $d=50$ and another sensitive attribute
• Figure 4: The strategic reaction of null and non-null agents in the market to regulations via testing. The above figures (b) and (c) assume the incentives in the market by fixing $C/R=0.15$

Theorems & Definitions (41)

• Definition 2.1: Credal Set
• Definition 2.2: Set of Marginally Desirable Gambles
• Definition 2.3: Social Choice Function
• Definition 2.4: Implementability
• Definition 3.1: Requirement
• Definition 3.2: Obedience
• Definition 3.3: Feasibility
• Definition 3.4
• Theorem 3.5
• Lemma 3.6