SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Compute

TL;DR

SecMoE addresses the challenge of privacy-preserving inference for large-scale Transformer models with MoE by introducing a Select-Then-Compute framework in a two-party setting. It unifies the MoE FFN and nonlinear activations into secure blocks and obviates evaluating all experts by privately selecting the active ones and computing only a single encrypted path, thereby preserving sparsity and greatly reducing communication. The key contributions include a Secure Sparse MoE protocol with obfuscated expert selection, a Select-Then-Compute approach for piecewise polynomial GeLU evaluation, and comprehensive complexity analyses and experiments showing substantial latency and bandwidth gains over prior 2‑PC PPML systems. The results demonstrate that SecMoE enables scalable, private MoE inference with up to 63× model capacity and significant end-to-end efficiency improvements in practical LAN/WAN settings. This work advances private inference toward real-world deployment of large-scale, sparsity-friendly Transformer models.

Abstract

Privacy-preserving Transformer inference has gained attention due to the potential leakage of private information. Despite recent progress, existing frameworks still fall short of practical model scales, with gaps up to a hundredfold. A possible way to close this gap is the Mixture of Experts (MoE) architecture, which has emerged as a promising technique to scale up model capacity with minimal overhead. However, given that the current secure two-party (2-PC) protocols allow the server to homomorphically compute the FFN layer with its plaintext model weight, under the MoE setting, this could reveal which expert is activated to the server, exposing token-level privacy about the client's input. While naively evaluating all the experts before selection could protect privacy, it nullifies MoE sparsity and incurs the heavy computational overhead that sparse MoE seeks to avoid. To address the privacy and efficiency limitations above, we propose a 2-PC privacy-preserving inference framework, \SecMoE. Unifying per-entry circuits in both the MoE layer and piecewise polynomial functions, \SecMoE obliviously selects the extracted parameters from circuits and only computes one encrypted entry, which we refer to as Select-Then-Compute. This makes the model for private inference scale to 63$\times$ larger while only having a 15.2$\times$ increase in end-to-end runtime. Extensive experiments show that, under 5 expert settings, \SecMoE lowers the end-to-end private inference communication by 1.8$\sim$7.1$\times$ and achieves 1.3$\sim$3.8$\times$ speedup compared to the state-of-the-art (SOTA) protocols.

Figures (3)

• Figure 1: An overview of $\mathsf{SecMoE}$, deploying Select-Then-Compute in Secure MoE FFN and secure approximation selection.
• Figure 2: Illustration of secure MoE layer. Comparison between a naive privacy-preserving MoE (left) and our proposed secure sparse MoE (right). The baseline evaluates all experts and aggregates their outputs via Softmax, leading to high overhead. In contrast, our method securely selects top experts and only evaluates one expert, significantly reducing computation and communication cost.
• Figure 3: The private MoE-Small (top) and Switch-Base (bottom) inference for Iron, BumbleBee, and $\mathsf{SecMoE}$. Begin with communication (MB) and time (s) under LAN (middle) and WAN (right) network settings.

Theorems & Definitions (1)

• Definition 1