SeSeMI: Secure Serverless Model Inference on Sensitive Data

TL;DR

SeSeMI tackles the problem of secure, scalable model inference on sensitive data in untrusted cloud environments by combining trusted hardware with a serverless architecture. Its three core components—KeyService for secure key management and access control, SeMIRT as a multi-request enclave runtime, and FnPacker for model packing and workload-aware routing—enable end-to-end encryption and efficient sharing of SGX enclaves. The approach achieves security, low latency, and reduced cost without intrusive changes to existing serverless platforms, demonstrated through extensive experiments on SGX-enabled clusters and multiple ML models. The work offers practical impact by enabling privacy-preserving, elastically scalable inference in production cloud settings and is open-sourced for reproducibility and adoption.

Abstract

Model inference systems are essential for implementing end-to-end data analytics pipelines that deliver the benefits of machine learning models to users. Existing cloud-based model inference systems are costly, not easy to scale, and must be trusted in handling the models and user request data. Serverless computing presents a new opportunity, as it provides elasticity and fine-grained pricing. Our goal is to design a serverless model inference system that protects models and user request data from untrusted cloud providers. It offers high performance and low cost, while requiring no intrusive changes to the current serverless platforms. To realize our goal, we leverage trusted hardware. We identify and address three challenges in using trusted hardware for serverless model inference. These challenges arise from the high-level abstraction of serverless computing, the performance overhead of trusted hardware, and the characteristics of model inference workloads. We present SeSeMI, a secure, efficient, and cost-effective serverless model inference system. It adds three novel features non-intrusively to the existing serverless infrastructure and nothing else.The first feature is a key service that establishes secure channels between the user and the serverless instances, which also provides access control to models and users' data. The second is an enclave runtime that allows one enclave to process multiple concurrent requests. The final feature is a model packer that allows multiple models to be executed by one serverless instance. We build SeSeMI on top of Apache OpenWhisk, and conduct extensive experiments with three popular machine learning models. The results show that SeSeMI achieves low latency and low cost at scale for realistic workloads.

Figures (18)

• Figure 1: Security issue of cloud model inference services.
• Figure 2: Example of serverless model inference.
• Figure 3: SeSeMI overview.
• Figure 4: Model serving stages inside a serverless instance.
• Figure 5: Enclave ecall & ocall APIs and Inference APIs in SeMIRT.