Single-Round Scalable Analytic Federated Learning

TL;DR

Federated Learning suffers from high communication costs and performance degradation on non-IID data. SAFLe introduces a non-linear analytic head built from bucketing feature spaces and sparse grouped embeddings, preserving AFL's single-round, gradient-free aggregation. The authors prove a high-dimensional linear-equivalence for the non-linear head, enabling exact solutions via Absolute Aggregation with Regularization Intermediary. Empirically, SAFLe achieves state-of-the-art accuracy among analytic FL approaches, outperforming AFL and DeepAFL on CIFAR-10/100 and Tiny-ImageNet while maintaining single-round efficiency and robustness to heterogeneity.

Abstract

Federated Learning (FL) is plagued by two key challenges: high communication overhead and performance collapse on heterogeneous (non-IID) data. Analytic FL (AFL) provides a single-round, data distribution invariant solution, but is limited to linear models. Subsequent non-linear approaches, like DeepAFL, regain accuracy but sacrifice the single-round benefit. In this work, we break this trade-off. We propose SAFLe, a framework that achieves scalable non-linear expressivity by introducing a structured head of bucketed features and sparse, grouped embeddings. We prove this non-linear architecture is mathematically equivalent to a high-dimensional linear regression. This key equivalence allows SAFLe to be solved with AFL's single-shot, invariant aggregation law. Empirically, SAFLe establishes a new state-of-the-art for analytic FL, significantly outperforming both linear AFL and multi-round DeepAFL in accuracy across all benchmarks, demonstrating a highly efficient and scalable solution for federated vision.

Figures (6)

• Figure 1: Illustration of the proposed SAFLe model. Input images are first processed by a pre-trained backbone network to extract features. A Pre-Non-Linearity by Bucketing transformation is then applied to each feature using one of three methods: Integer, One-Hot, or Binary Overlapping bucketing. The resulting bucketed features are shuffled and divided into $E$ groups, each of which is passed through a corresponding learnable embedding. The outputs of all embeddings are summed to produce the final model output. The embedding parameters admit an analytical solution and can be optimized through a federated learning algorithm that remains invariant to both data distribution and the number of participating clients.
• Figure 2: Accuracy vs. Communication Rounds on (a) CIFAR-100 and (b) Tiny-ImageNet. Analytic methods SAFLe and AFL achieve final accuracy in a single round.
• Figure 3: Accuracy vs. Total Communication Cost (MB) per client on (a) CIFAR-100 and (b) Tiny-ImageNet. Our single-round method, SAFLe, achieves a given accuracy (e.g., $\approx 62\%$ on Tiny-ImageNet) for a fraction of the total communication cost required by the multi-round DeepAFL.
• Figure 4: Comparison of accuracy and communication rounds for analytic federated methods. SAFLe (ours) achieves the highest accuracy with the lowest rounds.
• Figure 5: Accuracy over increasing numbers of clients $(K)$. The proposed SAFLe is mathematically invariant to the number of clients, similar to AFL, whereas FedAvg's performance declines as $K$ increases.

Theorems & Definitions (4)

• Lemma 1
• proof
• Theorem 1
• proof