AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data

TL;DR

AFCL addresses the challenge of spatio-temporal non-IID data in Federated Continual Learning by eliminating gradient-based updates and leveraging a gradient-free pipeline. It combines a frozen pre-trained backbone for feature extraction with a closed-form, regularized least-squares analytic classifier, enabling single-epoch local training and a single-round, recursive server aggregation that yields a global model equivalent to centralized ERM, i.e., $W_K = (\mathbf{F}_{1:K}^T\mathbf{F}_{1:K} + \gamma\mathbf{I})^{-1} \mathbf{F}_{1:K}^T \mathbf{Y}_{1:K}$. The approach introduces a known-unknown class splitting mechanism to handle class-incremental data and maintains a Global Knowledge Matrix that captures and propagates globally learned knowledge, with theoretical guarantees of spatio-temporal invariance and order invariance. Empirically, AFCL delivers consistent improvements across CIFAR-100, Tiny-ImageNet, and ImageNet-R under various spatio-temporal heterogeneity settings and achieves substantial efficiency gains by avoiding multiple gradient-based rounds of communication.

Abstract

Federated Continual Learning (FCL) enables distributed clients to collaboratively train a global model from online task streams in dynamic real-world scenarios. However, existing FCL methods face challenges of both spatial data heterogeneity among distributed clients and temporal data heterogeneity across online tasks. Such data heterogeneity significantly degrades the model performance with severe spatial-temporal catastrophic forgetting of local and past knowledge. In this paper, we identify that the root cause of this issue lies in the inherent vulnerability and sensitivity of gradients to non-IID data. To fundamentally address this issue, we propose a gradient-free method, named Analytic Federated Continual Learning (AFCL), by deriving analytical (i.e., closed-form) solutions from frozen extracted features. In local training, our AFCL enables single-epoch learning with only a lightweight forward-propagation process for each client. In global aggregation, the server can recursively and efficiently update the global model with single-round aggregation. Theoretical analyses validate that our AFCL achieves spatio-temporal invariance of non-IID data. This ideal property implies that, regardless of how heterogeneous the data are distributed across local clients and online tasks, the aggregated model of our AFCL remains invariant and identical to that of centralized joint learning. Extensive experiments show the consistent superiority of our AFCL over state-of-the-art baselines across various benchmark datasets and settings.

Figures (4)

• Figure 1: The spatial-temporal data heterogeneity and catastrophic forgetting in FCL.
• Figure 2: The framework of our proposed AFCL.
• Figure 3: Average accuracy of AFCL and the baselines among different tasks.
• Figure 5: The gradient update perspective of catastrophic forgetting with non-IID data in FCL.