Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge-Aware Evolution for Streaming Federated Continual Learning with Category Overlap and without Task Identifiers

Sixing Tan, Xianmin Liu

TL;DR

This work tackles streaming Federated Continual Learning where data batches are disjoint across FL rounds and may share categories, with no explicit task identifiers. It introduces FedKACE, a framework integrating adaptive inference model switching, gradient-balanced replay, and kernel spectral boundary buffer maintenance to resolve knowledge confusion and sustain performance across all observed categories. The authors provide theoretical regret analyses for the buffer maintenance component and the overall FedKACE pipeline, and show empirical gains on CIFAR100 and ImageNet100 over multiple overlap settings compared to strong baselines. The approach enables robust cross-client continual adaptation in realistic streaming federated settings, offering practical benefits for privacy-preserving, non-stationary learning without task boundaries. Overall, FedKACE advances practical streaming FCL by unifying adaptive model switching, dynamic replay weighting, and informed buffer construction with provable guarantees.

Abstract

Federated Continual Learning (FCL) leverages inter-client collaboration to balance new knowledge acquisition and prior knowledge retention in non-stationary data. However, existing batch-based FCL methods lack adaptability to streaming scenarios featuring category overlap between old and new data and absent task identifiers, leading to indistinguishability of old and new knowledge, uncertain task assignments for samples, and knowledge confusion.To address this, we propose streaming federated continual learning setting: per federated learning (FL) round, clients process streaming data with disjoint samples and potentially overlapping categories without task identifiers, necessitating sustained inference capability for all prior categories after each FL round.Next, we introduce FedKACE: 1) an adaptive inference model switching mechanism that enables unidirectional switching from local model to global model to achieve a trade-off between personalization and generalization; 2) a adaptive gradient-balanced replay scheme that reconciles new knowledge learning and old knowledge retention under overlapping-class scenarios; 3) a kernel spectral boundary buffer maintenance that preserves high-information and high-boundary-influence samples to optimize cross-round knowledge retention. Experiments across multiple scenarios and regret analysis demonstrate the effectiveness of FedKACE.

Knowledge-Aware Evolution for Streaming Federated Continual Learning with Category Overlap and without Task Identifiers

TL;DR

This work tackles streaming Federated Continual Learning where data batches are disjoint across FL rounds and may share categories, with no explicit task identifiers. It introduces FedKACE, a framework integrating adaptive inference model switching, gradient-balanced replay, and kernel spectral boundary buffer maintenance to resolve knowledge confusion and sustain performance across all observed categories. The authors provide theoretical regret analyses for the buffer maintenance component and the overall FedKACE pipeline, and show empirical gains on CIFAR100 and ImageNet100 over multiple overlap settings compared to strong baselines. The approach enables robust cross-client continual adaptation in realistic streaming federated settings, offering practical benefits for privacy-preserving, non-stationary learning without task boundaries. Overall, FedKACE advances practical streaming FCL by unifying adaptive model switching, dynamic replay weighting, and informed buffer construction with provable guarantees.

Abstract

Federated Continual Learning (FCL) leverages inter-client collaboration to balance new knowledge acquisition and prior knowledge retention in non-stationary data. However, existing batch-based FCL methods lack adaptability to streaming scenarios featuring category overlap between old and new data and absent task identifiers, leading to indistinguishability of old and new knowledge, uncertain task assignments for samples, and knowledge confusion.To address this, we propose streaming federated continual learning setting: per federated learning (FL) round, clients process streaming data with disjoint samples and potentially overlapping categories without task identifiers, necessitating sustained inference capability for all prior categories after each FL round.Next, we introduce FedKACE: 1) an adaptive inference model switching mechanism that enables unidirectional switching from local model to global model to achieve a trade-off between personalization and generalization; 2) a adaptive gradient-balanced replay scheme that reconciles new knowledge learning and old knowledge retention under overlapping-class scenarios; 3) a kernel spectral boundary buffer maintenance that preserves high-information and high-boundary-influence samples to optimize cross-round knowledge retention. Experiments across multiple scenarios and regret analysis demonstrate the effectiveness of FedKACE.
Paper Structure (30 sections, 7 theorems, 137 equations, 4 figures, 4 tables)

This paper contains 30 sections, 7 theorems, 137 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Federated Continual Learning
  4. Online Continual Learning
  5. Notation and Problem Formulation
  6. Methonology
  7. Adaptive Inference Model Switching Mechanism
  8. Adaptive Gradient-Balanced Replay Scheme
  9. Kernel Spectral Boundary Buffer Maintenance
  10. FedKACE
  11. Experiments
  12. Experiment Settings
  13. Baseline Experiments
  14. Ablation Studies
  15. Extend Research
  16. ...and 15 more sections

Key Result

Theorem 1

Under assumptions in Appendix Appendix C.1, with $\lambda_k^{t,j} \in \Omega_\lambda$, $\theta_k^{t,j} = \{h_k^{t,j}, \phi_k^{t,j}\} \in \Omega_\theta$ where $h^j$ and $\phi^j$ denote output layer and other network parameters respectively, client $k$ in FL round $t$ minimizes $L_{\text{total}}^{t}(\ for all $\theta_k^{t,j} \in \Omega_\theta$ and $\lambda_k^{t,j} \geq 0$. This saddle point conditio

Figures (4)

  • Figure 1: Illustration of the FedKACE framework. In FL round $t$, per Section \ref{['Section 4.2']}, client $k$ first trains the global model on New data and buffer samples to obtain a local model. Then, per Section \ref{['Section 4.3']}, client $k$ uses the local model to update its buffer. Afterwards, Client $k$ uploads the local model to the server for aggregation and receives the new global model. Finally, per Section \ref{['Section 4.1']}, the client evaluates the new global model on the updated buffer and decide whether to switch the inference model.
  • Figure 2: Trends in Method Accuracy and FedKACE Buffer Metrics Across FL Rounds(Tasks) on Cifar100.
  • Figure 3: Trends in Method Accuracy and FedKACE Buffer Metrics Across FL Rounds(Tasks) on ImageNet100.
  • Figure 4: Trends in Method Accuracy Across FL Rounds(Tasks).

Theorems & Definitions (7)

  • Theorem 1: Local Saddle Point Convergence of Adaptive Gradient-Balanced Replay Scheme
  • Theorem 2: Regret Upper Bound of Kernel Spectral Boundary Buffer Maintenance
  • Theorem 3: Regret Upper Bound of FedKACE
  • Theorem 1: Local Saddle Point Convergence of Adaptive Gradient-Balanced Replay Scheme
  • Theorem 2: Regret Upper Bound of Kernel Spectral Boundary Buffer Maintenance
  • Lemma 1: Recursive Regret Upper Bound for Local Training in FedKACE
  • Theorem 3: Regret Upper Bound of FedKACE