Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SVAFD: A Secure and Verifiable Co-Aggregation Protocol for Federated Distillation

Tian Wen, Sheng Sun, Yuwei Wang, Peiyan Chen, Zhiyuan Wu, Min Liu, Bo Gao

TL;DR

The paper tackles secure aggregation for Federated Distillation in heterogeneous, potentially adversarial settings. It introduces SVAFD, a verifiable co-aggregation protocol built on MM-LCC that separates knowledge filtration, aggregation, and verification, ensuring privacy, integrity, and verifiability even under a malicious server and colluding clients. The approach leverages CAL-based filtration, LCC encoding, bilinear pairing proofs, and a multi-party aggregation scheme to resist poisoning and inference attacks while maintaining efficiency. Experimental results across four FD architectures and standard datasets show improved accuracy and robust defense with acceptable system overhead, highlighting SVAFD's practical impact for secure federated, heterogeneous learning.

Abstract

Secure Aggregation (SA) is an indispensable component of Federated Learning (FL) that concentrates on privacy preservation while allowing for robust aggregation. However, most SA designs rely heavily on the unrealistic assumption of homogeneous model architectures. Federated Distillation (FD), which aggregates locally computed logits instead of model parameters, introduces a promising alternative for cooperative training in heterogeneous model settings. Nevertheless, we recognize two major challenges in implementing SA for FD. (i) Prior SA designs encourage a dominant server, who is solely responsible for collecting, aggregating and distributing. Such central authority facilitates server to forge aggregation proofs or collude to bypass the claimed security guarantees; (ii) Existing SA, tailored for FL models, overlook the intrinsic properties of logits, making them unsuitable for FD. To address these challenges, we propose SVAFD, the first SA protocol that is specifically designed for FD. At a high level, SVAFD incorporates two innovations: (i) a multilateral co-aggregation method tha redefines the responsibilities of clients and server. Clients autonomously evaluate and aggregate logits shares locally with a lightweight coding scheme, while the server handles ciphertext decoding and performs the task of generating verification proofs; (ii) a quality-aware knowledge filtration method that facilitates biased logits exclusion against poisoning attacks. Moreover, SVAFD is resilient to stragglers and colluding clients, making it well-suited for dynamic networks in real-world applications. We have implemented the SVAFD prototype over four emerging FD architectures and evaluated it against poisoning and inference attacks. Results demonstrate that SVAFD improves model accuracy, making it a significant step forward in secure and verifiable aggregation for heterogeneous FL systems.

SVAFD: A Secure and Verifiable Co-Aggregation Protocol for Federated Distillation

TL;DR

The paper tackles secure aggregation for Federated Distillation in heterogeneous, potentially adversarial settings. It introduces SVAFD, a verifiable co-aggregation protocol built on MM-LCC that separates knowledge filtration, aggregation, and verification, ensuring privacy, integrity, and verifiability even under a malicious server and colluding clients. The approach leverages CAL-based filtration, LCC encoding, bilinear pairing proofs, and a multi-party aggregation scheme to resist poisoning and inference attacks while maintaining efficiency. Experimental results across four FD architectures and standard datasets show improved accuracy and robust defense with acceptable system overhead, highlighting SVAFD's practical impact for secure federated, heterogeneous learning.

Abstract

Secure Aggregation (SA) is an indispensable component of Federated Learning (FL) that concentrates on privacy preservation while allowing for robust aggregation. However, most SA designs rely heavily on the unrealistic assumption of homogeneous model architectures. Federated Distillation (FD), which aggregates locally computed logits instead of model parameters, introduces a promising alternative for cooperative training in heterogeneous model settings. Nevertheless, we recognize two major challenges in implementing SA for FD. (i) Prior SA designs encourage a dominant server, who is solely responsible for collecting, aggregating and distributing. Such central authority facilitates server to forge aggregation proofs or collude to bypass the claimed security guarantees; (ii) Existing SA, tailored for FL models, overlook the intrinsic properties of logits, making them unsuitable for FD. To address these challenges, we propose SVAFD, the first SA protocol that is specifically designed for FD. At a high level, SVAFD incorporates two innovations: (i) a multilateral co-aggregation method tha redefines the responsibilities of clients and server. Clients autonomously evaluate and aggregate logits shares locally with a lightweight coding scheme, while the server handles ciphertext decoding and performs the task of generating verification proofs; (ii) a quality-aware knowledge filtration method that facilitates biased logits exclusion against poisoning attacks. Moreover, SVAFD is resilient to stragglers and colluding clients, making it well-suited for dynamic networks in real-world applications. We have implemented the SVAFD prototype over four emerging FD architectures and evaluated it against poisoning and inference attacks. Results demonstrate that SVAFD improves model accuracy, making it a significant step forward in secure and verifiable aggregation for heterogeneous FL systems.
Paper Structure (42 sections, 25 equations, 16 figures, 3 tables)

This paper contains 42 sections, 25 equations, 16 figures, 3 tables.

Table of Contents

  1. Introduction
  2. PRELIMINARY
  3. Overview of FD
  4. Lagrange Coding Computation
  5. Threat Model and Basic Requirements
  6. Challenges and Key Insights of SVAFD
  7. Challenges of Secure Aggreagation (SA)
  8. MM-LCC Design for Co-aggregation
  9. CAL for Quality-aware Knowledge Filtration
  10. IMPLEMENTATION OF SVAFD
  11. Framework Overview
  12. Knowledge Filtration
  13. Local Sensitive Hash
  14. Communication Topology
  15. Knowledge Aggregation
  16. ...and 27 more sections

Figures (16)

  • Figure 1: Federated Distillation Architecture and Security Threats. Note: The red box highlights that in FD, clients upload logits instead of models as in FL; the symbol "T" combined with a number denotes specific security threats, where threats "T1" and "T2" occur on the server side, while threat "T3" manifests on the client side.
  • Figure 2: Overview of SVAFD.
  • Figure 3: Workflow of LCC and MM-LCC.
  • Figure 4: Observation of CAL
  • Figure 5: Workflow of SVAFD.
  • ...and 11 more figures