Embedded Planogram Compliance Control System

TL;DR

The paper addresses planogram compliance in retail by proposing a fully embedded, edge-computing pipeline that integrates low-cost image capture, YOLOv5-based object detection, a modified Needleman-Wunsch sequence-alignment for planogram matching, and energy harvesting to enable stand-alone shelf monitoring. It demonstrates end-to-end feasibility on ESP-EYE cameras and multiple single-board computers (Raspberry Pi 4, NVIDIA Jetson Nano/AGX), achieving near-perfect object detection and planogram compliance (F1 scores of 0.997 and 1.0, respectively) on a Migros dataset. The approach scales across hardware configurations, with detailed timing and power analyses showing potential for years-long operation via solar and RF harvesting. The work contributes a practical, scalable solution for smart retail that reduces labor and improves shelf accuracy, with future directions toward lighter-weight models and broader AI-based alignment improvements.

Abstract

The retail sector presents several open and challenging problems that could benefit from advanced pattern recognition and computer vision techniques. One such critical challenge is planogram compliance control. In this study, we propose a complete embedded system to tackle this issue. Our system consists of four key components as image acquisition and transfer via stand-alone embedded camera module, object detection via computer vision and deep learning methods working on single board computers, planogram compliance control method again working on single board computers, and energy harvesting and power management block to accompany the embedded camera modules. The image acquisition and transfer block is implemented on the ESP-EYE camera module. The object detection block is based on YOLOv5 as the deep learning method and local feature extraction. We implement these methods on Raspberry Pi 4, NVIDIA Jetson Orin Nano, and NVIDIA Jetson AGX Orin as single board computers. The planogram compliance control block utilizes sequence alignment through a modified Needleman-Wunsch algorithm. This block is also working along with the object detection block on the same single board computers. The energy harvesting and power management block consists of solar and RF energy harvesting modules with suitable battery pack for operation. We tested the proposed embedded planogram compliance control system on two different datasets to provide valuable insights on its strengths and weaknesses. The results show that our method achieves F1 scores of 0.997 and 1.0 in object detection and planogram compliance control blocks, respectively. Furthermore, we calculated that the complete embedded system can work in stand-alone form up to two years based on battery. This duration can be further extended with the integration of the proposed solar and RF energy harvesting options.

Figures (3)

• Figure 1: Functional block diagram of the proposed system.
• Figure 2: YOLOv5 training results.
• Figure 3: Bounding box of the detected objects from the representative shelf images of test datasets.