Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PackVFL: Efficient HE Packing for Vertical Federated Learning

Liu Yang, Shuowei Cai, Di Chai, Junxue Zhang, Han Tian, Yilun Jin, Kun Guo, Kai Chen, Qiang Yang

TL;DR

PackVFL addresses the critical efficiency bottlenecks of HE-based vertical federated learning by employing PackedHE to enable SIMD-style computation and a domain-tailored MatMult design. The framework introduces a hybrid diagonal packing strategy, input packing/partitioning, and a lazy rotate-and-sum workflow, plus targeted adaptations for VFL-LinR, CAESAR, and VFL-NN. Empirical results show end-to-end speedups up to 51.52× over state-of-the-art HE-based VFL methods and significant reductions in rotation-based overhead without sacrificing model accuracy. These advances demonstrate that PackedHE can surpass Paillier-based approaches in both performance and security, enabling more scalable and practical VFL deployments.

Abstract

As an essential tool of secure distributed machine learning, vertical federated learning (VFL) based on homomorphic encryption (HE) suffers from severe efficiency problems due to data inflation and time-consuming operations. To this core, we propose PackVFL, an efficient VFL framework based on packed HE (PackedHE), to accelerate the existing HE-based VFL algorithms. PackVFL packs multiple cleartexts into one ciphertext and supports single-instruction-multiple-data (SIMD)-style parallelism. We focus on designing a high-performant matrix multiplication (MatMult) method since it takes up most of the ciphertext computation time in HE-based VFL. Besides, devising the MatMult method is also challenging for PackedHE because a slight difference in the packing way could predominantly affect its computation and communication costs. Without domain-specific design, directly applying SOTA MatMult methods is hard to achieve optimal. Therefore, we make a three-fold design: 1) we systematically explore the current design space of MatMult and quantify the complexity of existing approaches to provide guidance; 2) we propose a hybrid MatMult method according to the unique characteristics of VFL; 3) we adaptively apply our hybrid method in representative VFL algorithms, leveraging distinctive algorithmic properties to further improve efficiency. As the batch size, feature dimension and model size of VFL scale up to large sizes, PackVFL consistently delivers enhanced performance. Empirically, PackVFL propels existing VFL algorithms to new heights, achieving up to a 51.52X end-to-end speedup. This represents a substantial 34.51X greater speedup compared to the direct application of SOTA MatMult methods.

PackVFL: Efficient HE Packing for Vertical Federated Learning

TL;DR

PackVFL addresses the critical efficiency bottlenecks of HE-based vertical federated learning by employing PackedHE to enable SIMD-style computation and a domain-tailored MatMult design. The framework introduces a hybrid diagonal packing strategy, input packing/partitioning, and a lazy rotate-and-sum workflow, plus targeted adaptations for VFL-LinR, CAESAR, and VFL-NN. Empirical results show end-to-end speedups up to 51.52× over state-of-the-art HE-based VFL methods and significant reductions in rotation-based overhead without sacrificing model accuracy. These advances demonstrate that PackedHE can surpass Paillier-based approaches in both performance and security, enabling more scalable and practical VFL deployments.

Abstract

As an essential tool of secure distributed machine learning, vertical federated learning (VFL) based on homomorphic encryption (HE) suffers from severe efficiency problems due to data inflation and time-consuming operations. To this core, we propose PackVFL, an efficient VFL framework based on packed HE (PackedHE), to accelerate the existing HE-based VFL algorithms. PackVFL packs multiple cleartexts into one ciphertext and supports single-instruction-multiple-data (SIMD)-style parallelism. We focus on designing a high-performant matrix multiplication (MatMult) method since it takes up most of the ciphertext computation time in HE-based VFL. Besides, devising the MatMult method is also challenging for PackedHE because a slight difference in the packing way could predominantly affect its computation and communication costs. Without domain-specific design, directly applying SOTA MatMult methods is hard to achieve optimal. Therefore, we make a three-fold design: 1) we systematically explore the current design space of MatMult and quantify the complexity of existing approaches to provide guidance; 2) we propose a hybrid MatMult method according to the unique characteristics of VFL; 3) we adaptively apply our hybrid method in representative VFL algorithms, leveraging distinctive algorithmic properties to further improve efficiency. As the batch size, feature dimension and model size of VFL scale up to large sizes, PackVFL consistently delivers enhanced performance. Empirically, PackVFL propels existing VFL algorithms to new heights, achieving up to a 51.52X end-to-end speedup. This represents a substantial 34.51X greater speedup compared to the direct application of SOTA MatMult methods.
Paper Structure (43 sections, 8 equations, 7 figures, 5 tables, 4 algorithms)

This paper contains 43 sections, 8 equations, 7 figures, 5 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Our Techniques
  3. Systematical Exploration of Design Space.
  4. MatMult Design for VFL Characteristics.
  5. Adaption Design to SOTA VFL Algorithms.
  6. PackVFL's Innovations and New Insights.
  7. Other Related Work
  8. Background and Preliminaries
  9. Vertical Federated Learning
  10. HE-based VFL is a Widely-Adopted and Practical Solution.
  11. Paillier's Batching is Limited for VFL.
  12. Packed Homomorphic Encryption
  13. Designing PackedHE MatMult is Non-Trivial.
  14. Systematical Exploration of Design Space
  15. Slot Packing
  16. ...and 28 more sections

Figures (7)

  • Figure 1: Illustration of HE-based VFL. Some VFL protocols may contain a trusted third party, which we omit to simplify.
  • Figure 2: Illustration of slot packing methods, i.e., the row-order (naive) method, column-order method, and our proposed generalized diagonal method, for PackedHE MatMult operation $\bm{X}_A \llbracket \bm{y}_B \rrbracket = \llbracket \bm{X}_A \bm{y}_B \rrbracket$. We set $m = N' = 4, n = 2$.
  • Figure 3: Overview of PackVFL. PackVFL contains two parts: 1) we design a hybrid MatMult method in terms of the characteristics of VFL; 2) we adaptively apply the proposed MatMult method to representative VFL algorithms.
  • Figure 4: Illustration of PackVFL's diagonal method.
  • Figure 5: Illustration of adapting PackVFL' MatMult method to three SOTA VFL algorithms.
  • ...and 2 more figures