Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications

TL;DR

This survey provides a hardware‑driven synthesis of quantization for MCU‑class edge devices, connecting quantization theory with MCU architectures, NPUs, and software stacks. It systematically surveys PTQ, QAT, and advanced techniques (mixed/ extreme low‑bit quantization, hardware‑aware methods, and data‑agnostic approaches), emphasizing how granularity, calibration, and numerical representations affect real‑world deployment. The authors review ARM and RISC‑V MCU ecosystems, including NPU‑augmented platforms, and discuss practical deployments across healthcare, vision, and IoT domains, highlighting the trade‑offs between accuracy, latency, memory, and energy. They also outline open challenges—sub‑8‑bit support, on‑device training, mixed precision with static memory, and sustainability—and propose future directions for end‑to‑end co‑design of hardware, software, and data to enable scalable, energy‑efficient AI at the extreme edge.

Abstract

The deployment of Quantized Neural Networks (QNNs) on resource-constrained edge devices, such as microcontrollers (MCUs), introduces fundamental challenges in balancing model performance, computational complexity, and memory constraints. Tiny Machine Learning (TinyML) addresses these issues by jointly advancing machine learning algorithms, hardware architectures, and software optimization techniques to enable deep neural network inference on embedded systems. This survey provides a hardware-oriented perspective on neural network quantization, systematically reviewing the quantization methods most relevant to MCUs and extreme-edge devices. Particular emphasis is placed on the critical trade-offs between model performance and the capabilities of MCU-class hardware, including memory hierarchies, numerical representations, and accelerator support. The survey further reviews contemporary MCU hardware platforms, including ARM-based and RISC-V-based designs, as well as MCUs integrating neural processing units (NPUs) for low-precision inference, together with the supporting software stacks. In addition, we analyze real-world deployments of quantized models on MCUs and consolidate the application domains in which such systems are used. Finally, we discuss open challenges and outline promising future directions toward scalable, energy-efficient, and sustainable AI deployment on edge devices.

Figures (16)

• Figure 1: The taxonomy of the discussed topics in this survey.
• Figure 2: (Top) Representation of the FP32 format, (bottom left) signed INT8 format and (bottom right) unsigned INT8 format.
• Figure 3: Uniform quantization (left) vs. Non-uniform quantization (right). Real values $\mathbf{x}$ are mapped into lower precision integer values $\mathbf{x}_{\text{int}}$. The distances between the quantized values (quantization levels) are the same in uniform quantization, whereas they can vary in non-uniform quantization.
• Figure 4: Illustration of different quantization schemes: symmetric signed (left), symmetric unsigned (center), and asymmetric (right). Red markers denote the INT8 quantization range. The black labeled “0” on the real axis indicates the true zero in floating-point space, while the red zero shows its corresponding integer location after quantization. In symmetric quantization, the true zero and the mapped zero-point align, whereas in the asymmetric case, the zero-point is shifted.
• Figure 5: Simplified illustration of global quantization, where all weight matrices $W^{(1)}$, $W^{(2)}$, and $W^{(3)}$ across different layers share the same quantization parameters.