Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OLYMPIA: A Simulation Framework for Evaluating the Concrete Scalability of Secure Aggregation Protocols

Ivoline C. Ngong, Nicholas Gibson, Joseph P. Near

TL;DR

Olympia presents a simulation framework to enable end-to-end evaluation of secure aggregation protocols at scales unreachable by real hardware. It combines a Python-embedded DSL for protocol specification with an ABIDES-based simulator that models computation, network latency, bandwidth, and dropouts, providing realistic wall-clock timings. The authors implement multiple state-of-the-art protocols (including Bonawitz et al., Bell et al., Stevens et al., and ACORN) and perform a comprehensive empirical comparison across settings from hundreds to ten-thousand clients, revealing that computation often dominates total time and that server bandwidth can be a critical bottleneck. They validate Olympia’s accuracy against ground-truth deployments and show substantial speedups (e.g., simulating 10^4 clients on one machine) while offering new insights such as the limited impact of latency and the tangible gains from optimizations like packed secret sharing. The work provides open-source tooling and a standardized benchmark for future protocol development and deployment planning in large-scale secure aggregation for federated learning.

Abstract

Recent secure aggregation protocols enable privacy-preserving federated learning for high-dimensional models among thousands or even millions of participants. Due to the scale of these use cases, however, end-to-end empirical evaluation of these protocols is impossible. We present OLYMPIA, a framework for empirical evaluation of secure protocols via simulation. OLYMPIA. provides an embedded domain-specific language for defining protocols, and a simulation framework for evaluating their performance. We implement several recent secure aggregation protocols using OLYMPIA, and perform the first empirical comparison of their end-to-end running times. We release OLYMPIA as open source.

OLYMPIA: A Simulation Framework for Evaluating the Concrete Scalability of Secure Aggregation Protocols

TL;DR

Olympia presents a simulation framework to enable end-to-end evaluation of secure aggregation protocols at scales unreachable by real hardware. It combines a Python-embedded DSL for protocol specification with an ABIDES-based simulator that models computation, network latency, bandwidth, and dropouts, providing realistic wall-clock timings. The authors implement multiple state-of-the-art protocols (including Bonawitz et al., Bell et al., Stevens et al., and ACORN) and perform a comprehensive empirical comparison across settings from hundreds to ten-thousand clients, revealing that computation often dominates total time and that server bandwidth can be a critical bottleneck. They validate Olympia’s accuracy against ground-truth deployments and show substantial speedups (e.g., simulating 10^4 clients on one machine) while offering new insights such as the limited impact of latency and the tangible gains from optimizations like packed secret sharing. The work provides open-source tooling and a standardized benchmark for future protocol development and deployment planning in large-scale secure aggregation for federated learning.

Abstract

Recent secure aggregation protocols enable privacy-preserving federated learning for high-dimensional models among thousands or even millions of participants. Due to the scale of these use cases, however, end-to-end empirical evaluation of these protocols is impossible. We present OLYMPIA, a framework for empirical evaluation of secure protocols via simulation. OLYMPIA. provides an embedded domain-specific language for defining protocols, and a simulation framework for evaluating their performance. We implement several recent secure aggregation protocols using OLYMPIA, and perform the first empirical comparison of their end-to-end running times. We release OLYMPIA as open source.
Paper Structure (71 sections, 15 figures, 4 tables)

This paper contains 71 sections, 15 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Overview of Olympia
  4. Olympia: Evaluation via Simulation.
  5. Challenge: realism & scale.
  6. New Insights from Olympia.
  7. The Olympia Framework
  8. The Olympia DSL.
  9. The Olympia Simulator.
  10. The Olympia DSL
  11. Example: Baseline Insecure Summation Protocol.
  12. The Olympia Simulator
  13. Overview of simulation approach.
  14. Modeling computation time.
  15. Modeling network latency.
  16. ...and 56 more sections

Figures (15)

  • Figure 1: Olympia implementation of the baseline protocol.
  • Figure 2: Olympia implementation of the secret sharing protocol (client).
  • Figure 3: Olympia implementation of the secret sharing protocol (server).
  • Figure 4: Olympia implementation of the Stevens et al. stevens2022efficient protocol (client).
  • Figure 5: Total running time comparison between simulated and real experiments.
  • ...and 10 more figures