FairGRPO: Fair Reinforcement Learning for Equitable Clinical Reasoning

TL;DR

This work tackles demographic fairness in clinical reasoning by introducing FairGRPO, a fairness-aware hierarchical reinforcement learning method. FairGRPO extends Group Relative Policy Optimization with adaptive reward scaling across explicit and latent demographic groups, enabling equitable learning even when demographic labels are incomplete. Across 7 clinical datasets and 5 modalities, FairGRPO achieves substantial reductions in disparity (e.g., up to 27.2% predictive parity improvement) while also boosting F1 scores (≈12.5%), and it exhibits fairness improvements throughout training rather than degradation. The authors release FairMedGemma-4B, a fairness-aware clinical vision-language model, along with open training pipelines to support reproducible, equitable medical AI development.

Abstract

Medical artificial intelligence systems have achieved remarkable diagnostic capabilities, yet they consistently exhibit performance disparities across demographic groups, causing real-world harm to underrepresented populations. While recent multimodal reasoning foundation models have advanced clinical diagnosis through integrated analysis of diverse medical data, reasoning trainings via reinforcement learning inherit and often amplify biases present in training datasets dominated by majority populations. We introduce Fairness-aware Group Relative Policy Optimization (FairGRPO), a hierarchical reinforcement learning approach that promotes equitable learning across heterogeneous clinical populations. FairGRPO employs adaptive importance weighting of advantages based on representation, task difficulty, and data source. To address the common issue of missing demographic labels in the clinical domain, we further employ unsupervised clustering, which automatically discovers latent demographic groups when labels are unavailable. Through comprehensive experiments across 7 clinical diagnostic datasets spanning 5 clinical modalities across X-ray, CT scan, dermoscropy, mammography and ultrasound, we demonstrate that FairGRPO reduces predictive parity by 27.2% against all vanilla and bias mitigated RL baselines, while improving F1 score by 12.49%. Furthermore, training dynamics analysis reveals that FairGRPO progressively improves fairness throughout optimization, while baseline RL methods exhibit deteriorating fairness as training progresses. Based on FairGRPO, we release FairMedGemma-4B, a fairness-aware clinical VLLM that achieves state-of-the-art performance while demonstrating significantly reduced disparities across demographic groups.

Figures (4)

• Figure 1: FairGRPO Training Pipeline. Our method addresses fairness disparities by adaptively scaling rewards based on demographic representation and task difficulty. Starting with medical data containing both labeled demographic information and unlabeled samples, the policy model generates multiple responses for each prompt, producing both reasoning insights and clinical diagnoses. These responses are evaluated and assigned rewards. FairGRPO then groups the rewards by explicit demographic groups where available. For samples with unavailable demographic information, we employ K-means clustering to discover implicit groups. Then, minority or challenging groups receive amplified learning signals through inverse temperature scaling, while majority or well-represented groups are scaled down. This ensures that the model learns equitably from all subpopulations, preventing the typical bias toward majority groups that occurs in standard training.
• Figure 2: Training dynamics comparison between GRPO and FairGRPO on clinical classification tasks. (a) F1 Score: FairGRPO achieves higher F1 scores throughout training, reaching 0.265 compared to GRPO's plateau at 0.250. (b) Accuracy: Both methods converge to similar accuracy levels, with FairGRPO demonstrating slightly higher final accuracy. (c) F1 Diff: FairGRPO substantially reduces demographic performance disparities, achieving around 57% reduction in F1 difference by explicitly optimizing for fairness during training. (e) Per Step Runtime of the Models: We run the model using the setup described in Sec. \ref{['sec:experiment_setup']}. The reward calculation for all methods are less than 0.1% of the total runtime, showing it adds negligible overhead to the training process. (e) Performance-Fairness Tradeoff: We compare the validation F1 score and reversed F1 difference (1-F1 Diff) of different steps throughout a single training run. Pareto frontier is plotted to illustrate the points where the mdoel achieves the best tradeoff performance between F1 score and fairness. The starred point is the final model reported in Tab. \ref{['tab:fairness_performance']}. FairGRPO achieves superior Pareto optimality, simultaneously improving both performance and fairness compared to GRPO's best checkpoint.
• Figure 3: F1 score differences between FairGRPO and GRPO across demographic groups on MedGemma. Each bar represents the F1 score difference from the population mean for specific demographic subgroups, where a positive value means FairGRPO performs better for the given demographic group. The four age groups are binned as described in Sec. \ref{['sec:experiment_setup']}. In general, FairGRPO consistently demonstrates better performance for 25 out of the 33 demographic groups across datasets, which includes both majority and minority groups. Raw performance results are included in App. Tab. \ref{['tab:group_scores_raw']}.
• Figure 4: Qualitative Examples of Model's Reasoning Traces. We see the greatest performance boosts from underrepresented groups, including samples from older population and females. In particular, we observe the models trained with FairGRPO exhibit an accuracy improvement of 73.08% on 75+ populations in PAD-UFES-20 dataset, and a 36.53% on samples aged 51-75 in VinDr-Mammo. This figure shows examples of model's internal thinking process from the two groups.