Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection

Zhiwei Ling, Hailiang Zhao, Chao Zhang, Xiang Ao, Ziqi Wang, Cheng Zhang, Zhen Qin, Xinkui Zhao, Kingsum Chow, Yuanqing Wu, MengChu Zhou

TL;DR

Federated learning often struggles with non-IID client data, leading to unstable convergence and degraded generalization. The paper introduces FLood, a dual-weighting framework that leverages out-of-distribution (OOD) signals to guide both local training and global aggregation: at the client, pseudo-OOD samples receive upweighted loss, and at the server, client contributions are weighted by OOD confidence scores. The method combines Adaptive Sample Weighting with Dynamic Aggregation Correction, using a cosine-based schedule for weight growth and an aggregation rule that blends data volume with OOD-derived reliability, achieving a convergence guarantee of order $O(1/\sqrt{T})$ up to $(\epsilon+\delta)^2$ error. Extensive experiments on CIFAR-10/100 and SVHN under Dirichlet and Pathological non-IID settings show FLood consistently outperforms state-of-the-art FL methods and remains compatible as a plug-in module for existing algorithms, offering a practical boost for reliable edge-cloud intelligent services.

Abstract

Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes while preserving data privacy, making it a cornerstone of intelligent service systems in edge-cloud environments. However, in real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID. This severe data heterogeneity critically undermines the convergence stability, generalization ability, and ultimately the quality of service delivered by the global model. To address this challenge, we propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection. FLood dynamically counteracts the adverse effects of heterogeneity through a dual-weighting mechanism that jointly governs local training and global aggregation. At the client level, it adaptively reweights the supervised loss by upweighting pseudo-OOD samples, thereby encouraging more robust learning from distributionally misaligned or challenging data. At the server level, it refines model aggregation by weighting client contributions according to their OOD confidence scores, prioritizing updates from clients with higher in-distribution consistency and enhancing the global model's robustness and convergence stability. Extensive experiments across multiple benchmarks under diverse non-IID settings demonstrate that FLood consistently outperforms state-of-the-art FL methods in both accuracy and generalization. Furthermore, FLood functions as an orthogonal plug-in module: it seamlessly integrates with existing FL algorithms to boost their performance under heterogeneity without modifying their core optimization logic. These properties make FLood a practical and scalable solution for deploying reliable intelligent services in real-world federated environments.

Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection

TL;DR

Federated learning often struggles with non-IID client data, leading to unstable convergence and degraded generalization. The paper introduces FLood, a dual-weighting framework that leverages out-of-distribution (OOD) signals to guide both local training and global aggregation: at the client, pseudo-OOD samples receive upweighted loss, and at the server, client contributions are weighted by OOD confidence scores. The method combines Adaptive Sample Weighting with Dynamic Aggregation Correction, using a cosine-based schedule for weight growth and an aggregation rule that blends data volume with OOD-derived reliability, achieving a convergence guarantee of order up to error. Extensive experiments on CIFAR-10/100 and SVHN under Dirichlet and Pathological non-IID settings show FLood consistently outperforms state-of-the-art FL methods and remains compatible as a plug-in module for existing algorithms, offering a practical boost for reliable edge-cloud intelligent services.

Abstract

Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes while preserving data privacy, making it a cornerstone of intelligent service systems in edge-cloud environments. However, in real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID. This severe data heterogeneity critically undermines the convergence stability, generalization ability, and ultimately the quality of service delivered by the global model. To address this challenge, we propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection. FLood dynamically counteracts the adverse effects of heterogeneity through a dual-weighting mechanism that jointly governs local training and global aggregation. At the client level, it adaptively reweights the supervised loss by upweighting pseudo-OOD samples, thereby encouraging more robust learning from distributionally misaligned or challenging data. At the server level, it refines model aggregation by weighting client contributions according to their OOD confidence scores, prioritizing updates from clients with higher in-distribution consistency and enhancing the global model's robustness and convergence stability. Extensive experiments across multiple benchmarks under diverse non-IID settings demonstrate that FLood consistently outperforms state-of-the-art FL methods in both accuracy and generalization. Furthermore, FLood functions as an orthogonal plug-in module: it seamlessly integrates with existing FL algorithms to boost their performance under heterogeneity without modifying their core optimization logic. These properties make FLood a practical and scalable solution for deploying reliable intelligent services in real-world federated environments.
Paper Structure (27 sections, 1 theorem, 30 equations, 9 figures, 6 tables, 2 algorithms)

This paper contains 27 sections, 1 theorem, 30 equations, 9 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Learning
  4. Out-of-Distribution Detection
  5. Preliminaries
  6. FL Problem Setup
  7. Post-Hoc OOD Score
  8. The FLood Framework
  9. Overview
  10. Adaptive Sample Weighting
  11. Dynamic Aggregation Correction
  12. Convergence Analysis
  13. Experiments
  14. Setup
  15. Baselines
  16. ...and 12 more sections

Key Result

Theorem 1

Suppose the learning rate is set to $\eta = \frac{1}{\sqrt{T}}$. Then the iterates $\{\theta^t\}_{t=0}^{T-1}$ generated by FLood satisfy where $\epsilon$ and $\delta$ are defined in Assumptions ass:bias and ass:agg_bias.

Figures (9)

  • Figure 1: Key difference between FLood and FedAvg on the client side. The orange border highlights pseudo-OOD samples, while $\lambda$ denotes the reallocated weights assigned to these samples in the supervised loss.
  • Figure 2: Framework and workflow of FLood. The right illustrates the client-side training process, where OOD detection is used to adjust sample weights in the supervised loss. The left demonstrates the server-side process, where the server dynamically reallocates aggregation weights based on client-provided information.
  • Figure 3: Illustration of the weight adjustment curve in FLood. (a) Comparison of different weight scheduling functions, including cosine-based, linear, quadratic, exponential, and logistic schedules. (b) Weight growth under different starting phases.
  • Figure 4: Data distributions across clients under Dirichlet and Pathological non-i.i.d. partitioning protocols.
  • Figure 5: Accuracy curves of various FL methods on CIFAR-10 dataset under the $\text{Dir}(0.1)$ scenario. On the top, FLood is compared with FedAvg and server-side global adjustment methods, while on the bottom, it is compared with client-side local training correction methods.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Theorem 1: Convergence of FLood