Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set

Gordon Dai, Pavan Ravishankar, Rachel Yuan, Daniel B. Neill, Emily Black

TL;DR

This work investigates model multiplicity through the Rashomon set, the collection of all models with near-optimal accuracy, and its implications for fairness. It introduces precise theoretical and algorithmic tools: (i) the largest Rashomon set $R_N(\epsilon)$ and efficient sampling; (ii) efficient methods to find the fairest models under statistical parity and error-rate balance via knapsack-based optimizations; (iii) a closed-form large-sample expression for the probability that an individual’s prediction flips within the Rashomon set; (iv) a formula for the Rashomon-set size $|R_N(\epsilon)|$ that grows as $B(\epsilon)^N$; and (v) asymptotic results showing that, for large $N$, models in the Rashomon set tend to use the full error tolerance. Empirical results on German Credit, Adult, and Health datasets demonstrate substantial fairness gains from intentional LDA searches, reveal nontrivial flip-probability patterns across individuals, and illustrate how the Rashomon set’s size and error-tolerance usage scale with data size and heterogeneity. The work provides policy-relevant insights, recommending deliberate fairness searches within the Rashomon set and careful calibration of $\epsilon$ to balance fairness opportunities with arbitrariness risks.

Abstract

When selecting a model from a set of equally performant models, how much unfairness can you really reduce? Is it important to be intentional about fairness when choosing among this set, or is arbitrarily choosing among the set of ''good'' models good enough? Recent work has highlighted that the phenomenon of model multiplicity-where multiple models with nearly identical predictive accuracy exist for the same task-has both positive and negative implications for fairness, from strengthening the enforcement of civil rights law in AI systems to showcasing arbitrariness in AI decision-making. Despite the enormous implications of model multiplicity, there is little work that explores the properties of sets of equally accurate models, or Rashomon sets, in general. In this paper, we present five main theoretical and methodological contributions which help us to understand the relatively unexplored properties of the Rashomon set, in particular with regards to fairness. Our contributions include methods for efficiently sampling models from this set and techniques for identifying the fairest models according to key fairness metrics such as statistical parity. We also derive the probability that an individual's prediction will be flipped within the Rashomon set, as well as expressions for the set's size and the distribution of error tolerance used across models. These results lead to policy-relevant takeaways, such as the importance of intentionally looking for fair models within the Rashomon set, and understanding which individuals or groups may be more susceptible to arbitrary decisions.

Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set

TL;DR

This work investigates model multiplicity through the Rashomon set, the collection of all models with near-optimal accuracy, and its implications for fairness. It introduces precise theoretical and algorithmic tools: (i) the largest Rashomon set and efficient sampling; (ii) efficient methods to find the fairest models under statistical parity and error-rate balance via knapsack-based optimizations; (iii) a closed-form large-sample expression for the probability that an individual’s prediction flips within the Rashomon set; (iv) a formula for the Rashomon-set size that grows as ; and (v) asymptotic results showing that, for large , models in the Rashomon set tend to use the full error tolerance. Empirical results on German Credit, Adult, and Health datasets demonstrate substantial fairness gains from intentional LDA searches, reveal nontrivial flip-probability patterns across individuals, and illustrate how the Rashomon set’s size and error-tolerance usage scale with data size and heterogeneity. The work provides policy-relevant insights, recommending deliberate fairness searches within the Rashomon set and careful calibration of to balance fairness opportunities with arbitrariness risks.

Abstract

When selecting a model from a set of equally performant models, how much unfairness can you really reduce? Is it important to be intentional about fairness when choosing among this set, or is arbitrarily choosing among the set of ''good'' models good enough? Recent work has highlighted that the phenomenon of model multiplicity-where multiple models with nearly identical predictive accuracy exist for the same task-has both positive and negative implications for fairness, from strengthening the enforcement of civil rights law in AI systems to showcasing arbitrariness in AI decision-making. Despite the enormous implications of model multiplicity, there is little work that explores the properties of sets of equally accurate models, or Rashomon sets, in general. In this paper, we present five main theoretical and methodological contributions which help us to understand the relatively unexplored properties of the Rashomon set, in particular with regards to fairness. Our contributions include methods for efficiently sampling models from this set and techniques for identifying the fairest models according to key fairness metrics such as statistical parity. We also derive the probability that an individual's prediction will be flipped within the Rashomon set, as well as expressions for the set's size and the distribution of error tolerance used across models. These results lead to policy-relevant takeaways, such as the importance of intentionally looking for fair models within the Rashomon set, and understanding which individuals or groups may be more susceptible to arbitrary decisions.
Paper Structure (34 sections, 13 theorems, 62 equations, 15 figures, 3 algorithms)

This paper contains 34 sections, 13 theorems, 62 equations, 15 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work and Legal Background
  3. Preliminaries and Notation
  4. The Importance of Intentional Fairness
  5. Optimizing fairness over the Rashomon set
  6. Optimizing for statistical parity
  7. Optimizing for error rate balance
  8. Sampling models uniformly at random from the Rashomon set
  9. Experiments on real data
  10. Baseline approach: sampling linear models from the Rashomon set
  11. Takeaways for policy and practice
  12. Understanding Individual Flip Probabilities
  13. Preliminaries and assumptions for our large-sample theoretical results
  14. Individual flip probabilities
  15. Experiments on real data
  16. ...and 19 more sections

Key Result

Theorem 5.1

Given the preliminaries and assumptions above, as $N\rightarrow\infty$, the flip probability corresponding to a data record with weight $w_i = w$ converges to where $C(\epsilon) = g^{-1}(\epsilon)$ and $g(C) = \int_0^1 \frac{w f(w)}{1+\exp(Cw)} \, dw$.

Figures (15)

  • Figure 1: Disparity in positive prediction rate for the German, Adult, and Health datasets, as a function of the error tolerance $\epsilon$. Comparison of methods for optimizing PPR (Section \ref{['sec:optimizing-PPR']}), uniform random sampling (Section \ref{['sec:sampling']}), and sampling linear models (Section \ref{['sec:linear']}) over the Rashomon set $R_N(\epsilon)$.
  • Figure 2: Left: Flip probability $q_{N,i}$ as a function of the Bayes-optimal probability $p_i$--- in other words, how likely is an individual $i$ to experience a change of prediction among models in the Rashomon set as a function of their true probability that $y_i = 1$? We show results for the German Credit, Adult, and Health datasets for $\epsilon\in \{0.001, 0.01, 0.02\}$, and see that there is large variation in flip probability distribution both as a function of dataset and $\epsilon$. Center: Overall (population average) flip probability as a function of error tolerance $\epsilon$ for the German Credit dataset, for uniformly sampled models, linear models, and optimally fair models from the Rashomon set. For results for Adult and Health datasets, see Appendix \ref{['appendix:flip_probs']}, Figure \ref{['fig:appendix_overall_flip_probs']}. Right: Group average flip probability, comparison between protected group (solid lines) and non-protected group (dashed lines), for the German Credit dataset, as a function of the error tolerance $\epsilon$. Comparison of methods for optimizing PPR, FPR, and TPR (Section \ref{['sec:optimizing-fairness']}) and uniform random sampling (Section \ref{['sec:sampling']}), over the Rashomon set $R_N(\epsilon)$. For results for Adult and Health datasets, see Appendix \ref{['appendix:flip_probs']}, Figure \ref{['fig:appendix_stratified_flip_probs']}.
  • Figure 3: Left: Rashomon set size as a function of $\epsilon$ for Adult, German Credit, and Health datasets, and for uniformly distributed weights. Note that the German Credit and uniform weights curves coincide. The size of the Rashomon set is $|R_N(\epsilon)| = B(\epsilon)^N$, where the exponential base $B$ (plotted here) ranges between 1 (for $\epsilon = 0$) and 2 (for large $\epsilon$). We also separately plot $|R_N(\epsilon)|$ for each dataset in Appendix \ref{['appendix:rashomon_set_size-experiments']}, Figure \ref{['fig:appendix_Rashomon_set_size']}. Right three figures: Proportion of error tolerance used, $\frac{\theta \cdot W_N}{N\epsilon}$, for the German, Adult, and Health datasets, as a function of the error tolerance $\epsilon$. Comparison of methods for optimizing PPR (Section \ref{['sec:optimizing-PPR']}), optimizing FPR (Section \ref{['sec:optimizing-TPR and FPR']}), optimizing TPR (Section \ref{['sec:optimizing-TPR and FPR']}), uniform random sampling (Section \ref{['sec:sampling']}), and sampling linear models (Section \ref{['sec:linear']}) over the Rashomon set $R_N(\epsilon)$.
  • Figure 4: Disparity in false positive rate for the German, Adult, and Health datasets, as a function of the error tolerance $\epsilon$. Comparison of methods for optimizing FPR (Section \ref{['sec:optimizing-TPR and FPR']}), uniform random sampling (Section \ref{['sec:sampling']}), and sampling linear models (Section \ref{['sec:linear']}) over the Rashomon set $R_N(\epsilon)$.
  • Figure 5: Disparity in true positive rate for the German, Adult, and Health datasets, as a function of the error tolerance $\epsilon$. Comparison of methods for optimizing TPR (Section \ref{['sec:optimizing-TPR and FPR']}), uniform random sampling (Section \ref{['sec:sampling']}), and sampling linear models (Section \ref{['sec:linear']}) over the Rashomon set $R_N(\epsilon)$.
  • ...and 10 more figures

Theorems & Definitions (29)

  • Theorem 5.1: Asymptotic flip probabilities
  • Theorem 6.1: Asymptotic size of Rashomon set
  • Theorem 6.2: Asymptotic use of the entire error tolerance
  • definition 1: Accuracy of a model defined by a flip vector $\theta$
  • definition 2: Rashomon set
  • definition 3: Flip probability
  • lemma 1: Relationship between flip probability, weight, and Rashomon set size
  • proof
  • lemma 2: Asymptotic Pairwise Independence of Flip Probabilities
  • proof
  • ...and 19 more