Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Invariant Federated Learning for Edge Intelligence: Mitigating Heterogeneity and Asynchrony via Exit Strategy and Invariant Penalty

Ziruo Hao, Zhenhua Cui, Tao Yang, Bo Hu, Xiaofeng Wu, Hui Feng

TL;DR

This work tackles heterogeneity and asynchrony in edge federated learning by introducing an exit strategy for aberrant clients and a FedIPG framework that enforces parameter invariance via a penalty term grounded in parameter orthogonality. By decomposing client parameters into invariant $w^I$ and heterogeneous $w^e_i$ components and minimizing $R_i(w) + \lambda \|\langle \nabla R_i(w), w \rangle\|^2$, the approach promotes stable global learning and improved OOD generalization. The authors provide theoretical analyses on contributions, generalization bounds, and convergence, and validate the method on multiple datasets with varying scales, demonstrating enhanced ID and OOD performance and hints of causal robustness. The proposed exit mechanism also reallocates communication toward better-equipped clients, improving convergence speed without increasing communication overhead. Overall, FedIPG offers a practical, theory-backed solution for robust edge FL under heterogeneity and asynchronous environments.

Abstract

This paper provides an invariant federated learning system for resource-constrained edge intelligence. This framework can mitigate the impact of heterogeneity and asynchrony via exit strategy and invariant penalty. We introduce parameter orthogonality into edge intelligence to measure the contribution or impact of heterogeneous and asynchronous clients. It is proved in this paper that the exit of abnormal edge clients can guarantee the effect of the model on most clients. Meanwhile, to ensure the models' performance on exited abnormal clients and those who lack training resources, we propose Federated Learning with Invariant Penalty for Generalization (FedIPG) by constructing the approximate orthogonality of the invariant parameters and the heterogeneous parameters. Theoretical proof shows that FedIPG reduces the Out-Of-Distribution prediction loss without increasing the communication burden. The performance of FedIPG combined with an exit strategy is tested empirically in multiple scales using four datasets. It shows our system can enhance In-Distribution performance and outperform the state-of-the-art algorithm in Out-Of-Distribution generalization while maintaining model convergence. Additionally, the results of the visual experiment prove that FedIPG contains preliminary causality in terms of ignoring confounding features.

Invariant Federated Learning for Edge Intelligence: Mitigating Heterogeneity and Asynchrony via Exit Strategy and Invariant Penalty

TL;DR

This work tackles heterogeneity and asynchrony in edge federated learning by introducing an exit strategy for aberrant clients and a FedIPG framework that enforces parameter invariance via a penalty term grounded in parameter orthogonality. By decomposing client parameters into invariant and heterogeneous components and minimizing , the approach promotes stable global learning and improved OOD generalization. The authors provide theoretical analyses on contributions, generalization bounds, and convergence, and validate the method on multiple datasets with varying scales, demonstrating enhanced ID and OOD performance and hints of causal robustness. The proposed exit mechanism also reallocates communication toward better-equipped clients, improving convergence speed without increasing communication overhead. Overall, FedIPG offers a practical, theory-backed solution for robust edge FL under heterogeneity and asynchronous environments.

Abstract

This paper provides an invariant federated learning system for resource-constrained edge intelligence. This framework can mitigate the impact of heterogeneity and asynchrony via exit strategy and invariant penalty. We introduce parameter orthogonality into edge intelligence to measure the contribution or impact of heterogeneous and asynchronous clients. It is proved in this paper that the exit of abnormal edge clients can guarantee the effect of the model on most clients. Meanwhile, to ensure the models' performance on exited abnormal clients and those who lack training resources, we propose Federated Learning with Invariant Penalty for Generalization (FedIPG) by constructing the approximate orthogonality of the invariant parameters and the heterogeneous parameters. Theoretical proof shows that FedIPG reduces the Out-Of-Distribution prediction loss without increasing the communication burden. The performance of FedIPG combined with an exit strategy is tested empirically in multiple scales using four datasets. It shows our system can enhance In-Distribution performance and outperform the state-of-the-art algorithm in Out-Of-Distribution generalization while maintaining model convergence. Additionally, the results of the visual experiment prove that FedIPG contains preliminary causality in terms of ignoring confounding features.

Paper Structure

This paper contains 16 sections, 6 theorems, 55 equations, 7 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. The System Model of Invariant Federated Learning
  4. The Federated System Model with Exit Strategy
  5. The Profile of the Parameter Orthogonality the Invariant Schemes
  6. The basis of Aberrant Clients Exit Strategy
  7. Theoretical Measure for Clients' Contribution
  8. Contribution Analysis for Aberrant clients
  9. Generalization and Convergence Analysis of FedIPG
  10. Out-of-Distributed Generalization by Invariant Penalty Term
  11. Convergence Analysis of Invariant Asynchronous Federated Learning Algorithm
  12. Simulation Results
  13. Exit Strategy for Aberrant Clients
  14. Leave-One-Domain-Out Generalization
  15. Anti-Confounding Causal Generalization
  16. ...and 1 more sections

Key Result

Theorem 1

(The Contribution Analysis for Heterogeneous client) For a optimization started from the origin of coordinates, while $w^t=\kappa \frac{w^I}{\|w^I\|} + \nu \frac{w^e_i}{\|w^e_i\|}$, $\kappa\in (0,1)$ and $\nu \in (-\epsilon,\epsilon)$, the contribution of client $i$ in iteration $t$ can be represent

Figures (7)

  • Figure 1: Edge intelligence scenarios with diverse computing and communication capabilities
  • Figure 2: The process of federated learning in edge intelligence scenarios with guaranteed OOD generalization
  • Figure 3: Trend of accuracy and loss function changes as the degree of anomaly changes
  • Figure 4: Average test accuracy with different hyperparameter $\lambda$.
  • Figure 5: Train and test accuracy using causal validation with different quality of confounding in CNN network.
  • ...and 2 more figures

Theorems & Definitions (9)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • proof
  • Theorem 5
  • proof
  • Theorem 6
  • proof