Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Memory-Efficient and Secure DNN Inference on TrustZone-enabled Consumer IoT Devices

Xueshuo Xie, Haoxu Wang, Zhaolong Jian, Tao Li, Wei Wang, Zhiwei Xu, Guiling Wang

TL;DR

A novel approach for advanced model deployment in TrustZone that ensures comprehensive privacy preservation during model inference is presented, and a memory-efficient management method to support memory-demanding inference in TEEs is designed.

Abstract

Edge intelligence enables resource-demanding Deep Neural Network (DNN) inference without transferring original data, addressing concerns about data privacy in consumer Internet of Things (IoT) devices. For privacy-sensitive applications, deploying models in hardware-isolated trusted execution environments (TEEs) becomes essential. However, the limited secure memory in TEEs poses challenges for deploying DNN inference, and alternative techniques like model partitioning and offloading introduce performance degradation and security issues. In this paper, we present a novel approach for advanced model deployment in TrustZone that ensures comprehensive privacy preservation during model inference. We design a memory-efficient management method to support memory-demanding inference in TEEs. By adjusting the memory priority, we effectively mitigate memory leakage risks and memory overlap conflicts, resulting in 32 lines of code alterations in the trusted operating system. Additionally, we leverage two tiny libraries: S-Tinylib (2,538 LoCs), a tiny deep learning library, and Tinylibm (827 LoCs), a tiny math library, to support efficient inference in TEEs. We implemented a prototype on Raspberry Pi 3B+ and evaluated it using three well-known lightweight DNN models. The experimental results demonstrate that our design significantly improves inference speed by 3.13 times and reduces power consumption by over 66.5% compared to non-memory optimization method in TEEs.

Memory-Efficient and Secure DNN Inference on TrustZone-enabled Consumer IoT Devices

TL;DR

A novel approach for advanced model deployment in TrustZone that ensures comprehensive privacy preservation during model inference is presented, and a memory-efficient management method to support memory-demanding inference in TEEs is designed.

Abstract

Edge intelligence enables resource-demanding Deep Neural Network (DNN) inference without transferring original data, addressing concerns about data privacy in consumer Internet of Things (IoT) devices. For privacy-sensitive applications, deploying models in hardware-isolated trusted execution environments (TEEs) becomes essential. However, the limited secure memory in TEEs poses challenges for deploying DNN inference, and alternative techniques like model partitioning and offloading introduce performance degradation and security issues. In this paper, we present a novel approach for advanced model deployment in TrustZone that ensures comprehensive privacy preservation during model inference. We design a memory-efficient management method to support memory-demanding inference in TEEs. By adjusting the memory priority, we effectively mitigate memory leakage risks and memory overlap conflicts, resulting in 32 lines of code alterations in the trusted operating system. Additionally, we leverage two tiny libraries: S-Tinylib (2,538 LoCs), a tiny deep learning library, and Tinylibm (827 LoCs), a tiny math library, to support efficient inference in TEEs. We implemented a prototype on Raspberry Pi 3B+ and evaluated it using three well-known lightweight DNN models. The experimental results demonstrate that our design significantly improves inference speed by 3.13 times and reduces power consumption by over 66.5% compared to non-memory optimization method in TEEs.
Paper Structure (15 sections, 3 equations, 9 figures, 5 tables)

This paper contains 15 sections, 3 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. ARM TrustZone
  4. Deep Learning Framework
  5. Secure DNN inference in TEEs
  6. Smart-Zone
  7. Desin Principle
  8. Threat Model.
  9. On-demand Memory Management
  10. Tinylib Supported DNN inference
  11. Evaluation
  12. Experimental Setup
  13. Performance Analysis
  14. Comparing with SOTA Works
  15. Conclusion

Figures (9)

  • Figure 1: The Design Overview of Smart-Zone.
  • Figure 2: The mapping between virtual memory and physical memory through page table.
  • Figure 3: The invoke operation flow optimization.
  • Figure 4: The inference flow chart of Tinylib.
  • Figure 5: The performance on computation flow optimization of Tinylib at the model building stage.
  • ...and 4 more figures