Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Orion: A Fully Homomorphic Encryption Framework for Deep Learning

Austin Ebel, Karthik Garimella, Brandon Reagen

TL;DR

Orion tackles the core barriers to privacy-preserving deep learning with fully homomorphic encryption by automating translation of PyTorch models into CKKS-based FHE programs. It introduces a single-shot multiplexed convolution packing and an automated, linear-time bootstrap-placement algorithm, enabling deeper and larger networks to run in FHE with substantially fewer bootstraps and rotations than prior work. The framework integrates a PyTorch-like API, scale-management, and automated parameter handling, and demonstrates state-of-the-art single-threaded latency on ResNet-20 and large-scale object detection with YOLO-v1 on ImageNet-scale models, including a 139M-parameter backbone. These contributions significantly advance practical, end-to-end private neural inference, lowering the barrier to experimentation and deployment in privacy-sensitive cloud settings.

Abstract

Fully Homomorphic Encryption (FHE) has the potential to substantially improve privacy and security by enabling computation directly on encrypted data. This is especially true with deep learning, as today, many popular user services are powered by neural networks in the cloud. Beyond its well-known high computational costs, one of the major challenges facing wide-scale deployment of FHE-secured neural inference is effectively mapping these networks to FHE primitives. FHE poses many programming challenges including packing large vectors, managing accumulated noise, and translating arbitrary and general-purpose programs to the limited instruction set provided by FHE. These challenges make building large FHE neural networks intractable using the tools available today. In this paper we address these challenges with Orion, a fully-automated framework for private neural inference using FHE. Orion accepts deep neural networks written in PyTorch and translates them into efficient FHE programs. We achieve this by proposing a novel single-shot multiplexed packing strategy for arbitrary convolutions and through a new, efficient technique to automate bootstrap placement and scale management. We evaluate Orion on common benchmarks used by the FHE deep learning community and outperform state-of-the-art by 2.38x on ResNet-20, the largest network they report. Orion's techniques enable processing much deeper and larger networks. We demonstrate this by evaluating ResNet-50 on ImageNet and present the first high-resolution FHE object detection experiments using a YOLO-v1 model with 139 million parameters. Orion is open-source for all to use at: https://github.com/baahl-nyu/orion

Orion: A Fully Homomorphic Encryption Framework for Deep Learning

TL;DR

Orion tackles the core barriers to privacy-preserving deep learning with fully homomorphic encryption by automating translation of PyTorch models into CKKS-based FHE programs. It introduces a single-shot multiplexed convolution packing and an automated, linear-time bootstrap-placement algorithm, enabling deeper and larger networks to run in FHE with substantially fewer bootstraps and rotations than prior work. The framework integrates a PyTorch-like API, scale-management, and automated parameter handling, and demonstrates state-of-the-art single-threaded latency on ResNet-20 and large-scale object detection with YOLO-v1 on ImageNet-scale models, including a 139M-parameter backbone. These contributions significantly advance practical, end-to-end private neural inference, lowering the barrier to experimentation and deployment in privacy-sensitive cloud settings.

Abstract

Fully Homomorphic Encryption (FHE) has the potential to substantially improve privacy and security by enabling computation directly on encrypted data. This is especially true with deep learning, as today, many popular user services are powered by neural networks in the cloud. Beyond its well-known high computational costs, one of the major challenges facing wide-scale deployment of FHE-secured neural inference is effectively mapping these networks to FHE primitives. FHE poses many programming challenges including packing large vectors, managing accumulated noise, and translating arbitrary and general-purpose programs to the limited instruction set provided by FHE. These challenges make building large FHE neural networks intractable using the tools available today. In this paper we address these challenges with Orion, a fully-automated framework for private neural inference using FHE. Orion accepts deep neural networks written in PyTorch and translates them into efficient FHE programs. We achieve this by proposing a novel single-shot multiplexed packing strategy for arbitrary convolutions and through a new, efficient technique to automate bootstrap placement and scale management. We evaluate Orion on common benchmarks used by the FHE deep learning community and outperform state-of-the-art by 2.38x on ResNet-20, the largest network they report. Orion's techniques enable processing much deeper and larger networks. We demonstrate this by evaluating ResNet-50 on ImageNet and present the first high-resolution FHE object detection experiments using a YOLO-v1 model with 139 million parameters. Orion is open-source for all to use at: https://github.com/baahl-nyu/orion
Paper Structure (33 sections, 1 equation, 10 figures, 5 tables, 1 algorithm)

This paper contains 33 sections, 1 equation, 10 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. CKKS Background
  3. Datatypes
  4. Encoding: Cleartext $\rightarrow$ Plaintext
  5. Encryption: Plaintext $\rightarrow$ Ciphertext
  6. RNS-CKKS
  7. CKKS Operations
  8. Addition
  9. Multiplication
  10. Rotation
  11. Bootstrapping
  12. FHE Matrix-Vector Products
  13. Diagonal Encoding Method
  14. Baby-Step Giant-Step Optimization
  15. Hoisting Optimizations
  16. ...and 18 more sections

Figures (10)

  • Figure 1: $\mathsf{PMult}$
  • Figure 2: $\mathsf{HRot}$
  • Figure 3: $\mathsf{Bootstrap}$
  • Figure 5: Visualizing how the BSGS algorithm halevishoup reduces the number of ciphertext rotations in matrix-vector products.
  • Figure 6: An example showing how we transform the packed SISO method from Gazelle (b) into its analogous Toeplitz matrix (a) to leverage the BSGS and hoisting optimizations.
  • ...and 5 more figures