Federated Active Learning Under Extreme Non-IID and Global Class Imbalance

TL;DR

A systematic study of query-model selection in FAL uncovers a central insight: the model that achieves more class-balanced sampling, especially for minority classes, consistently leads to better final performance.

Abstract

Federated active learning (FAL) seeks to reduce annotation cost under privacy constraints, yet its effectiveness degrades in realistic settings with severe global class imbalance and highly heterogeneous clients. We conduct a systematic study of query-model selection in FAL and uncover a central insight: the model that achieves more class-balanced sampling, especially for minority classes, consistently leads to better final performance. Moreover, global-model querying is beneficial only when the global distribution is highly imbalanced and client data are relatively homogeneous; otherwise, the local model is preferable. Based on these findings, we propose FairFAL, an adaptive class-fair FAL framework. FairFAL (1) infers global imbalance and local-global divergence via lightweight prediction discrepancy, enabling adaptive selection between global and local query models; (2) performs prototype-guided pseudo-labeling using global features to promote class-aware querying; and (3) applies a two-stage uncertainty-diversity balanced sampling strategy with k-center refinement. Experiments on five benchmarks show that FairFAL consistently outperforms state-of-the-art approaches under challenging long-tailed and non-IID settings. The code is available at https://github.com/chenchenzong/FairFAL.

Figures (6)

• Figure 1: Comparison of global (G) and local (L) query selectors under varying $(\alpha,\rho)$ configurations on CIFAR-10. Test accuracy is plotted against the proportion of labeled samples for Entropy and Coreset sampling.
• Figure 2: Cumulative sampling proportions of majority, intermediate, and minority classes for global (G) and local (L) query selectors under different $(\alpha,\rho)$ settings on CIFAR-10, where the three groups denote the top 3, middle 4, and bottom 3 classes by sample proportion.
• Figure 3: Test accuracy vs. proportion of labeled samples on FMNIST, CIFAR-10, and CIFAR-100 under global imbalance $\rho=20$ and two heterogeneity levels ($\alpha=0.1$ and $\alpha=100$). Each subplot corresponds to a dataset–heterogeneity setting, and curves compare FairFAL with FAL and AL baselines using global (G-) or local (L-) models. Numerical results are provided in Tables \ref{['tab:main_numeric']} and \ref{['tab:main_numeric2']} in the Appendix.
• Figure 4: Visualization of the data distributions under the four $(\alpha,\rho)$ configurations discussed in the Observation section. These plots illustrate how client heterogeneity and global class imbalance jointly shape the federated data landscape.
• Figure 5: Test accuracy versus labeled-data ratio on CIFAR-10 under global imbalance ($\rho{=}20$) and two heterogeneity levels ($\alpha{=}0.1$ and $\alpha{=}100$), evaluated with FedProx and SCAFFOLD. Each subplot shows the corresponding FL framework and heterogeneity setting, comparing FairFAL with FAL and AL baselines using either the global (G-) or local (L-) query model.