VeriSplit: Secure and Practical Offloading of Machine Learning Inferences across IoT Devices

TL;DR

The paper tackles the challenge of offloading deep ML inferences from budget IoT devices to locally available devices without compromising user privacy, model secrets, or results integrity. It introduces VeriSplit, a framework that uses linear input masking for data privacy, mask-based parameter confidentiality with two non-colluding workers, and a commitment-based verification scheme built on Merkle trees to separate verification from the inference path and enable asynchronous/partial checks. Unlike prior cryptographic or TEEs-based approaches, VeriSplit preserves floating-point model fidelity and avoids special hardware, achieving practical latency improvements of up to $83\%$ over local execution in diverse tests, including Vision Transformers and CNNs. The work demonstrates that secure, private, and practical cross-device ML offloading is achievable in home IoT settings, enabling cost savings and better utilization of idle local compute resources.

Abstract

Many Internet-of-Things (IoT) devices rely on cloud computation resources to perform machine learning inferences. This is expensive and may raise privacy concerns for users. Consumers of these devices often have hardware such as gaming consoles and PCs with graphics accelerators that are capable of performing these computations, which may be left idle for significant periods of time. While this presents a compelling potential alternative to cloud offloading, concerns about the integrity of inferences, the confidentiality of model parameters, and the privacy of users' data mean that device vendors may be hesitant to offload their inferences to a platform managed by another manufacturer. We propose VeriSplit, a framework for offloading machine learning inferences to locally-available devices that address these concerns. We introduce masking techniques to protect data privacy and model confidentiality, and a commitment-based verification protocol to address integrity. Unlike much prior work aimed at addressing these issues, our approach does not rely on computation over finite field elements, which may interfere with floating-point computation supports on hardware accelerators and require modification to existing models. We implemented a prototype of VeriSplit and our evaluation results show that, compared to performing computation locally, our secure and private offloading solution can reduce inference latency by 28%--83%.

Figures (10)

• Figure 1: VeriSplit deployment example in a smart home. The IoT device (a smart camera) can offload ML inferences to one or more local devices by sharing input data and model parameters () and receiving results (). Later, the camera can verify the integrity of the inference by repeating selected computations and comparing the results (), doing it asynchronously to avoid additional latency during inference.
• Figure 2: VeriSplit's workflow for data privacy. During setup, the device shares model parameters with the worker, generates multiple one-time masks $\epsilon_i$, and precomputes their values $W \epsilon_i$. For each inference, the device sends masked input $x+\epsilon_i$ and then subtracts the precomputed values from the received result $y'$ to recover actual computation results.
• Figure 3: A simple but ineffective approach to adding masks for model confidentiality. This does not save any work for the device during inference, since it still needs to compute $\delta x_i$.
• Figure 4: VeriSplit's workflow for model confidentiality. During setup, the offloading device generates masking noises ($\delta, \beta$) and shares modified model parameters with two non-colluding workers. For inferences, the device sends data and receives layer results ($y'_1, y'_2$). It can reconstruct the actual results by combining the two results.
• Figure 5: VeriSplit's inference commitment design based on Merkle trees. This example network includes several 2D convolution layers, one flatten and one dense layer. During inference, the worker computes hash values of all layers' intermediate results ($h_1$--$h_8$) and reduces them into the final commit value $h_{\mathrm{commit}}$. Assuming the verifier randomly decides to check results from layer 3 (blue shadow), the worker sends all values in blue boxes as its integrity proof. This commitment mechanism can be expanded to partially verify layer results.

Theorems & Definitions (13)

• Definition 7.1: $\epsilon$-Masking Schemes
• Definition 7.2: Secure Masking Schemes
• Definition 7.3: Masking Scheme Failures
• Theorem 7.1: Perfectly Secure Masking Schemes
• Theorem 7.2: Failure Rates for $\epsilon$-Masking
• Corollary 7.2.1: Estimated Failure Rates for Data Privacy and Model Confidentiality
• Definition 7.4: $\alpha$-Verified Offload
• Theorem 7.3: Failure Rates for $\alpha$-Verification over $k$ Iterations
• proof : Proof of \ref{['thm:perfect-secure-masking']}
• proof : Proof of \ref{['thm:failure-rates-masking']}