OOS-DSD: Improving Out-of-stock Detection in Retail Images using Auxiliary Tasks

TL;DR

This work tackles the problem of detecting out-of-stock conditions in retail shelf images. It introduces OOS-DSD, a multitask extension of YOLOv8 that jointly performs OOS detection, product segmentation, and scene depth estimation, using pseudo-depth labels from Depth Anything V2 and a depth normalization step. The main contributions are the three-task architecture, the depth normalization technique, and comprehensive ablations showing that auxiliary learning yields up to a $3.7\%$ mAP gain (with depth normalization contributing up to $4.2\%$) and that the method achieves at least $1.8\%$ higher mAP than state-of-the-art methods. These results demonstrate the effectiveness of multitask learning for improving OOS detection in retail imagery and point to future extensions in multiview setups for enhanced scene understanding.

Abstract

Out-of-stock (OOS) detection is a very important retail verification process that aims to infer the unavailability of products in their designated areas on the shelf. In this paper, we introduce OOS-DSD, a novel deep learning-based method that advances OOS detection through auxiliary learning. In particular, we extend a well-established YOLOv8 object detection architecture with additional convolutional branches to simultaneously detect OOS, segment products, and estimate scene depth. While OOS detection and product segmentation branches are trained using ground truth data, the depth estimation branch is trained using pseudo-labeled annotations produced by the state-of-the-art (SOTA) depth estimation model Depth Anything V2. Furthermore, since the aforementioned pseudo-labeled depth estimates display relative depth, we propose an appropriate depth normalization procedure that stabilizes the training process. The experimental results show that the proposed method surpassed the performance of the SOTA OOS detection methods by 1.8% of the mean average precision (mAP). In addition, ablation studies confirm the effectiveness of auxiliary learning and the proposed depth normalization procedure, with the former increasing mAP by 3.7% and the latter by 4.2%.

Figures (5)

• Figure 1: The main idea behind OOS-DSD. The method relies on auxiliary learning to enhance OOS detection performance, with auxiliary tasks being product semantic segmentation and scene depth estimation.
• Figure 2: An example of an image and annotations for each of the addressed tasks: (a) shelf image and OOS bounding boxes, (b) segmentation map, and (c) depth map. In (a), normal and front OOS classes are marked with red and blue bounding boxes, respectively.
• Figure 3: Discrepancy between relative depth estimates of shelf images that captured scenes of different absolute depth. The columns display: (a) shelf images, (b) depth maps produced by Depth Anything V2, and (c) normalized depth maps.
• Figure 4: Scheme of the proposed multitask architecture and utilized blocks: (a) overall architecture, (b) Conv block, (c) Bottleneck block, (d) C2f block, (e) SPPF block, (f) Segment/Depth heads, and (g) Detect head. In (a), each block is marked with a number shown above the block. Blocks 0-22 are the same as in the YOLOv8 model. T and F stand for true and false, respectively.
• Figure 5: Examples of the ground truth data and predictions by the OOS-DSD for two shelf images: (a, f) OOS detection results, (b, g) ground truth segmentation maps, (c, h) predicted segmentation maps, (d, i) normalized (pseudo-ground truth) depth maps, and (e, j) predicted depth maps. Output results of the two images are grouped (a-e) and (f-j). In (a, f), red color marks the normal OOS class, blue color marks the front OOS class, full line denotes accurate detection, and dashed line denotes not recognized OOS.