Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Zero Rotation and Beyond: Architecting Neural Networks for Fast Secure Inference with Homomorphic Encryption

Yifei Cai, Yizhou Feng, Qiao Zhang, Chunsheng Xin, Hongyi Wu

TL;DR

This paper addresses the inefficiency of privacy-preserving CNN inference under Homomorphic Encryption in MLaaS. It proposes StriaNet, built from StriaBlock with an exRot-Free convolution and Cross Kernel, guided by two architectural principles to minimize HE rotation costs while preserving capacity. The approach yields substantial end-to-end speedups (e.g., $9.78\times$ on ImageNet, $6.01\times$ on Tiny ImageNet, and $9.24\times$ on CIFAR-10) and reduces communication overhead compared to alternatives like Cheetah, demonstrating the viability of HE-aware neural architecture design. These results suggest a practical path toward fast, secure inference and motivate further exploration via neural architecture search in HE settings.

Abstract

Privacy-preserving deep learning addresses privacy concerns in Machine Learning as a Service (MLaaS) by using Homomorphic Encryption (HE) for linear computations. However, the computational overhead remains a major challenge. While prior work has improved efficiency, most approaches build on models originally designed for plaintext inference. Such models incur architectural inefficiencies when adapted to HE. We argue that substantial gains require networks tailored to HE rather than retrofitting plaintext architectures. Our design has two components: the building block and the overall architecture. First, StriaBlock targets the most expensive HE operation, rotation. It integrates ExRot-Free Convolution and a novel Cross Kernel, eliminating external rotations and requiring only 19% of the internal rotations used by plaintext models. Second, our architectural principles include (i) the Focused Constraint Principle, which limits cost-sensitive factors while preserving flexibility elsewhere, and (ii) the Channel Packing-Aware Scaling Principle, which adapts bottleneck ratios to ciphertext channel capacity that varies with depth. Together, these strategies control both local and end-to-end HE cost, enabling a balanced HE-tailored network. We evaluate the resulting StriaNet across datasets of varying scales, including ImageNet, Tiny ImageNet, and CIFAR-10. At comparable accuracy, StriaNet achieves speedups of 9.78x, 6.01x, and 9.24x on ImageNet, Tiny ImageNet, and CIFAR-10, respectively.

Towards Zero Rotation and Beyond: Architecting Neural Networks for Fast Secure Inference with Homomorphic Encryption

TL;DR

This paper addresses the inefficiency of privacy-preserving CNN inference under Homomorphic Encryption in MLaaS. It proposes StriaNet, built from StriaBlock with an exRot-Free convolution and Cross Kernel, guided by two architectural principles to minimize HE rotation costs while preserving capacity. The approach yields substantial end-to-end speedups (e.g., on ImageNet, on Tiny ImageNet, and on CIFAR-10) and reduces communication overhead compared to alternatives like Cheetah, demonstrating the viability of HE-aware neural architecture design. These results suggest a practical path toward fast, secure inference and motivate further exploration via neural architecture search in HE settings.

Abstract

Privacy-preserving deep learning addresses privacy concerns in Machine Learning as a Service (MLaaS) by using Homomorphic Encryption (HE) for linear computations. However, the computational overhead remains a major challenge. While prior work has improved efficiency, most approaches build on models originally designed for plaintext inference. Such models incur architectural inefficiencies when adapted to HE. We argue that substantial gains require networks tailored to HE rather than retrofitting plaintext architectures. Our design has two components: the building block and the overall architecture. First, StriaBlock targets the most expensive HE operation, rotation. It integrates ExRot-Free Convolution and a novel Cross Kernel, eliminating external rotations and requiring only 19% of the internal rotations used by plaintext models. Second, our architectural principles include (i) the Focused Constraint Principle, which limits cost-sensitive factors while preserving flexibility elsewhere, and (ii) the Channel Packing-Aware Scaling Principle, which adapts bottleneck ratios to ciphertext channel capacity that varies with depth. Together, these strategies control both local and end-to-end HE cost, enabling a balanced HE-tailored network. We evaluate the resulting StriaNet across datasets of varying scales, including ImageNet, Tiny ImageNet, and CIFAR-10. At comparable accuracy, StriaNet achieves speedups of 9.78x, 6.01x, and 9.24x on ImageNet, Tiny ImageNet, and CIFAR-10, respectively.
Paper Structure (23 sections, 4 equations, 15 figures, 14 tables)

This paper contains 23 sections, 4 equations, 15 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. System Model
  4. Threat Model and Security Analysis
  5. Packed HE
  6. StriaNet: HE-Efficient Deep Learning Network and Architecting Methodology
  7. Building Block Design
  8. exRot-Free Pattern in MIMO
  9. Cross Kernel in SISO
  10. StriaBlock
  11. HE-Efficient Architectural Principle
  12. Network Architecture HE Complexity Analysis
  13. Channel Packing Aware Network Scaling
  14. Comparing with Depthwise Convolution
  15. Performance Evaluation
  16. ...and 8 more sections

Figures (15)

  • Figure 1: Performance of StriaNet across ImageNet, Tiny ImageNe and CIFAR-10 datasets.
  • Figure 2: Privacy-preserving MLaaS.
  • Figure 3: Grouped Out-Rot MIMO ($c_i=c_o=4,c_n =2$).
  • Figure 4: StriaBlock with exRot-Free Pattern ($c_i=c_o=16, c_n =4$).
  • Figure 5: SISO ($W=H=3,k_w=k_h=3$).
  • ...and 10 more figures