AEGIS: Scaling Long-Sequence Homomorphic Encrypted Transformer Inference via Hybrid Parallelism on Multi-GPU Systems

Abstract

Fully Homomorphic Encryption (FHE) enables privacy-preserving Transformer inference, but long-sequence encrypted Transformers quickly exceed single-GPU memory capacity because encoded weights are already large and encrypted activations grow rapidly with sequence length. Multi-GPU execution therefore becomes unavoidable, yet scaling remains challenging because communication is jointly induced by application-level aggregation and encryption-level RNS coupling. Existing approaches either synchronize between devices frequently or replicate encrypted tensors across devices, leading to excessive communication and latency. We present AEGIS, an Application-Encryption Guided Inference System for scalable long-sequence encrypted Transformer inference on multi-GPU platforms. AEGIS derives device placement from ciphertext dependencies jointly induced by Transformer dataflow and CKKS polynomial coupling, co-locating modulus-coherent and token-coherent data so that communication is introduced only when application dependencies require it, while reordering polynomial operators to overlap the remaining collectives with computation. On 2048-token inputs, AEGIS reduces inter-GPU communication by up to 57.9% in feed-forward networks and 81.3% in self-attention versus prior state-of-the-art designs. On four GPUs, it achieves up to 96.62% scaling efficiency, 3.86x end-to-end speedup, and 69.1% per-device memory reduction. These results establish coordinated application-encryption parallelism as a practical foundation for scalable homomorphic Transformer inference.

Figures (13)

• Figure 1: Illustration of data layout and parallelization models in CKKS computation. (a) Packing that maps plaintext tensors into ciphertext slots. (b) Coefficient-parallel, distributing work across polynomial coefficients. (c) Limb-parallel, distributing work across RNS primes and exposing thread parallel on coefficient.
• Figure 2: Encrypted Transformer inference pipeline with bootstrapping placement (NEXUS zhang_secure_2025).
• Figure 3: Fundamental parallelisms in plaintext inference. (a) Data parallelism replicates model weights across devices. (b,c) Tensor parallelism partitions weights and aggregates partial results via collectives.
• Figure 4: Memory breakdown of encrypted BERT-Base inference across input sequence lengths. The stacked bars represent the major memory components. Common NVIDIA GPU memory capacities are marked for reference.
• Figure 5: (a) Collective in RNS-parallel BSGS matrix multiplication down to compute graph level. (b) Latency breakdown of ciphertext matrix multiplication on a 2-GPU with NVLink.