FRAPPE: A Group Fairness Framework for Post-Processing Everything

TL;DR

FRAPPÉ addresses the practical constraints of achieving group fairness by converting regularized in-processing objectives into post-processing modules. It introduces an additive post-hoc correction $T_{ ext{PP}}(X)$ so that $f_{ ext{fair}}(X)=f_{ ext{base}}(X)+T_{ ext{PP}}(X)$ and trains $T_{ ext{PP}}$ via a bi-level objective that couples a discrepancy measure with a fairness regularizer, without retraining the base model or needing sensitive attributes at inference. Theoretical results show an equivalence between IP and PP objectives for GLMs, implying identical fairness-accuracy Pareto frontiers, while extensive experiments on Adult, COMPAS, HSLS, ENEM, and Communities & Crime demonstrate that FRAPPÉ can match or surpass in-processing trade-offs and outperform many post-processing baselines, even with continuous sensitive attributes and partial group labels. The modular design offers computational efficiency and broad applicability across definitions of fairness, problem settings, and model classes, enabling post-processing to be a practical tool in resource-constrained or multi-component systems.

Abstract

Despite achieving promising fairness-error trade-offs, in-processing mitigation techniques for group fairness cannot be employed in numerous practical applications with limited computation resources or no access to the training pipeline of the prediction model. In these situations, post-processing is a viable alternative. However, current methods are tailored to specific problem settings and fairness definitions and hence, are not as broadly applicable as in-processing. In this work, we propose a framework that turns any regularized in-processing method into a post-processing approach. This procedure prescribes a way to obtain post-processing techniques for a much broader range of problem settings than the prior post-processing literature. We show theoretically and through extensive experiments that our framework preserves the good fairness-error trade-offs achieved with in-processing and can improve over the effectiveness of prior post-processing methods. Finally, we demonstrate several advantages of a modular mitigation strategy that disentangles the training of the prediction model from the fairness mitigation, including better performance on tasks with partial group labels.

Figures (20)

• Figure 1: Inference with $\texttt{FRAPP\'E}$ and in-processing.$\texttt{FRAPP\'E}$ methods add the output of post-hoc module $T_\text{PP}$ to the unfair scores output by pre-trained model $f_\text{base}$. Unlike prior post-processing methods, $\texttt{FRAPP\'E}$ does not require sensitive attributes for inference. While in-processing trains the entire prediction model $f_{IP}$ to induce fairness, $\texttt{FRAPP\'E}$ only trains the post-hoc module. Note that, for classification, thresholding the predicted scores yields outputs $\hat{Y}$, while for regression $\hat{Y}$ coincides with the score.
• Figure 2: $\texttt{FRAPP\'E}$ and in-processing training objectives. Unlike existing post-processing techniques, $\texttt{FRAPP\'E}$ methods can be trained with any in-processing fairness regularizer $\mathcal{L}_{fair}$ (orange box). In contrast to in-processing, $\texttt{FRAPP\'E}$ only trains the post-hoc module $T_\text{PP}(X)$ instead of the entire prediction model $f$. Loss terms are computed on data that is labeled, unlabeled or annotated with sensitive attributes, as indicated. $d_\text{pred}$ measures the difference between the outputs of the base and the fair models (see \ref{['sec:method']}).
• Figure 3: Inducing three different definition of fairness (EqOpp, SP, and EqOdds) using in-processing methods and their $\texttt{FRAPP\'E}$ post-processing variant leads to similar Pareto frontiers. Thanks to their modular design, $\texttt{FRAPP\'E}$ methods only need to retrain the post-hoc transformation $T_\text{PP}(x)$, instead of the entire prediction model. \ref{['appendix:ip_vs_pp']} shows similar results on the Adult, COMPAS and ENEM datasets. Notably, $\texttt{FRAPP\'E}$mary19 is the first post-processing method that can operate on data with continuous sensitive attributes, such as Communities & Crime.
• Figure 4: Comparison between $\texttt{FRAPP\'E}$ MinDiff for EqOdds and the best-performing post-processing method alghamdi22, for random forest pre-trained models. See \ref{['fig:app_more_baselines']} for a comparison with more post-processing baselines. While in-processing MinDiff cannot be used with non-gradient based models, $\texttt{FRAPP\'E}$ MinDiff performs on-par or better than competitive post-processing approaches such as alghamdi22, even when the post-hoc transformation is as simple as linear regression or a 1-MLP.
• Figure 5: In-processing MinDiff and $\texttt{FRAPP\'E}$ MinDiff with partial group labels on the Adult dataset with optimal early-stopping (ES) regularization. Our post-processing algorithm continues to perform well even in the extreme case where in-processing cannot outperform the trivial baseline described in \ref{['sec:novel_failure']}.

Theorems & Definitions (3)

• Proposition 3.0
• Proposition 1.0
• proof