Federated Computing -- Survey on Building Blocks, Extensions and Systems

TL;DR

This paper defines Federated Computing (FC) as the union of Federated Learning (FL) and Federated Analytics (FA) and presents a comprehensive, expandable framework to describe FC systems. It dissects FC into core building blocks (hardware, client selection, aggregation, communication) and extensions (privacy and compression), and introduces a meta layer capturing motivation and hardware environment. By surveying over 150 papers, the authors distinguish FL and FA, map their system configurations, and highlight gaps—most FL work focuses on model performance with single-node or unknown environments, while FA papers emphasize privacy extensions and one-round analytics. The taxonomy and detailed system-oriented analysis enable standardized comparisons, identify bottlenecks, and guide reproducible and scalable design for privacy-preserving distributed computation in practice.

Abstract

In response to the increasing volume and sensitivity of data, traditional centralized computing models face challenges, such as data security breaches and regulatory hurdles. Federated Computing (FC) addresses these concerns by enabling collaborative processing without compromising individual data privacy. This is achieved through a decentralized network of devices, each retaining control over its data, while participating in collective computations. The motivation behind FC extends beyond technical considerations to encompass societal implications. As the need for responsible AI and ethical data practices intensifies, FC aligns with the principles of user empowerment and data sovereignty. FC comprises of Federated Learning (FL) and Federated Analytics (FA). FC systems became more complex over time and they currently lack a clear definition and taxonomy describing its moving pieces. Current surveys capture domain-specific FL use cases, describe individual components in an FC pipeline individually or decoupled from each other, or provide a quantitative overview of the number of published papers. This work surveys more than 150 papers to distill the underlying structure of FC systems with their basic building blocks, extensions, architecture, environment, and motivation. We capture FL and FA systems individually and point out unique difference between those two.

Figures (10)

• Figure 1: Intersection of definitions for FC, FL, and FA. FC consists of FL and FA, whereas FL systems can contain FA characteristics. FA is a subset of FL and FL is a subset of FC.
• Figure 2: Three FC architectures (centralized, hierarchical and peer-to-peer) with different aggregation server locations. Illustration inspired by he2020FedML. A node can be a client, a server or both, which is indicated by the circle filling.
• Figure 3: Interaction between basic building blocks for FC systems and optional extensions. The basic building blocks consist of hardware (devices hosting data set and an aggregation server) and software components (client selection, aggregation strategy and communication protocol).
• Figure 4: Framework to describe how different building blocks in an FC system work together. The illustrated pipeline focuses on FC basic building blocks and it shows some example options for each block. Additionally, it shows some options for the systems meta information.
• Figure 5: Two Pareto trade-offs between data transparency and privacy. Everything on the curve represents a privacy preserving outcome. The level of privacy for Point B is in both cases the same, but the level of transparency increased in the right trade-off due to privacy-enhancing techniques trask2020paretopriyanshu2021_2_single_manual.