Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HW-SW Optimization of DNNs for Privacy-preserving People Counting on Low-resolution Infrared Arrays

Matteo Risso, Chen Xie, Francesco Daghero, Alessio Burrello, Seyedmorteza Mollaei, Marco Castellano, Enrico Macii, Massimo Poncino, Daniele Jahier Pagliari

TL;DR

This work tackles privacy-preserving people counting from low-resolution infrared sensors by introducing a fully automated full-stack optimization flow that jointly searches DNN architectures (via PIT mask-based differentiable NAS), applies mixed-precision quantization (INT4/INT8 with QAT), and adds a lightweight temporal post-processing technique. It couples this software strategy with MAUPITI, a low-power edge platform built around a customized RISC-V IBEX core capable of efficient 4×4/8×8 vectorized DNN operations, delivering a wide Pareto frontier across energy, memory, and accuracy. Key contributions include the architecture/search framework, hardware-aware quantization with a restricted but effective precision scheme, a zero-additional-memory post-processing approach, and a hardware-software deployment toolchain validated on the LINAIGE IR dataset with substantial improvements over the state-of-the-art (e.g., up to 4.2× memory reduction, 23.8× code-size reduction, and 15.38× energy reduction at iso-accuracy). The results demonstrate practical edge deployment potential for privacy-safe occupancy sensing, highlighting significant end-to-end efficiency gains and freedom from high-resolution visual data.

Abstract

Low-resolution infrared (IR) array sensors enable people counting applications such as monitoring the occupancy of spaces and people flows while preserving privacy and minimizing energy consumption. Deep Neural Networks (DNNs) have been shown to be well-suited to process these sensor data in an accurate and efficient manner. Nevertheless, the space of DNNs' architectures is huge and its manual exploration is burdensome and often leads to sub-optimal solutions. To overcome this problem, in this work, we propose a highly automated full-stack optimization flow for DNNs that goes from neural architecture search, mixed-precision quantization, and post-processing, down to the realization of a new smart sensor prototype, including a Microcontroller with a customized instruction set. Integrating these cross-layer optimizations, we obtain a large set of Pareto-optimal solutions in the 3D-space of energy, memory, and accuracy. Deploying such solutions on our hardware platform, we improve the state-of-the-art achieving up to 4.2x model size reduction, 23.8x code size reduction, and 15.38x energy reduction at iso-accuracy.

HW-SW Optimization of DNNs for Privacy-preserving People Counting on Low-resolution Infrared Arrays

TL;DR

This work tackles privacy-preserving people counting from low-resolution infrared sensors by introducing a fully automated full-stack optimization flow that jointly searches DNN architectures (via PIT mask-based differentiable NAS), applies mixed-precision quantization (INT4/INT8 with QAT), and adds a lightweight temporal post-processing technique. It couples this software strategy with MAUPITI, a low-power edge platform built around a customized RISC-V IBEX core capable of efficient 4×4/8×8 vectorized DNN operations, delivering a wide Pareto frontier across energy, memory, and accuracy. Key contributions include the architecture/search framework, hardware-aware quantization with a restricted but effective precision scheme, a zero-additional-memory post-processing approach, and a hardware-software deployment toolchain validated on the LINAIGE IR dataset with substantial improvements over the state-of-the-art (e.g., up to 4.2× memory reduction, 23.8× code-size reduction, and 15.38× energy reduction at iso-accuracy). The results demonstrate practical edge deployment potential for privacy-safe occupancy sensing, highlighting significant end-to-end efficiency gains and freedom from high-resolution visual data.

Abstract

Low-resolution infrared (IR) array sensors enable people counting applications such as monitoring the occupancy of spaces and people flows while preserving privacy and minimizing energy consumption. Deep Neural Networks (DNNs) have been shown to be well-suited to process these sensor data in an accurate and efficient manner. Nevertheless, the space of DNNs' architectures is huge and its manual exploration is burdensome and often leads to sub-optimal solutions. To overcome this problem, in this work, we propose a highly automated full-stack optimization flow for DNNs that goes from neural architecture search, mixed-precision quantization, and post-processing, down to the realization of a new smart sensor prototype, including a Microcontroller with a customized instruction set. Integrating these cross-layer optimizations, we obtain a large set of Pareto-optimal solutions in the 3D-space of energy, memory, and accuracy. Deploying such solutions on our hardware platform, we improve the state-of-the-art achieving up to 4.2x model size reduction, 23.8x code size reduction, and 15.38x energy reduction at iso-accuracy.
Paper Structure (21 sections, 3 equations, 7 figures, 1 table)

This paper contains 21 sections, 3 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Neural Architecture Search (NAS)
  4. Quantization and Mixed-Precision
  5. Privacy-Preserving People Counting
  6. Methods
  7. Software Optimization Flow
  8. Architecture Optimization
  9. Precision Optimization
  10. Post-Processing
  11. The MAUPITI Platform
  12. Complete System
  13. Core Customization
  14. Deployment Toolchain
  15. Experimental Results
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Overview of the full-stack optimization flow.
  • Figure 2: Left: The PIT mask-based DNAS scheme. Right: three example results from the search procedure on a layer with four filters.
  • Figure 3: The complete MAUPITI System.
  • Figure 4: Customized IBEX RISC-V core.
  • Figure 5: Architecture and Precision Search Space exploration results. Different colors encode the different precisions' configurations.
  • ...and 2 more figures