Analyzing the Shopping Journey: Computing Shelf Browsing Visits in a Physical Retail Store

TL;DR

The paper addresses the challenge of understanding shopper intent in physical stores to enable autonomous robots. It introduces shelf visits, a trajectory-based measure of browsing derived from overhead-camera 3D tracking, and a shelf-stop algorithm calibrated with two human-labeled datasets ($N_1=8138$ and $N_2=15129$ trajectories). Cross-store evaluation demonstrates generalization, with $F_1$ scores around $0.84$–$0.89$ when calibrating on one store and testing on the other. Beyond method development, the work analyzes browsing patterns and their relationship to actual purchases, and discusses applications for retail planning and human-robot interaction, including potential real-time guidance and targeted recommendations.

Abstract

Motivated by recent challenges in the deployment of robots into customer-facing roles within retail, this work introduces a study of customer activity in physical stores as a step toward autonomous understanding of shopper intent. We introduce an algorithm that computes shoppers' ``shelf visits'' -- capturing their browsing behavior in the store. Shelf visits are extracted from trajectories obtained via machine vision-based 3D tracking and overhead cameras. We perform two independent calibrations of the shelf visit algorithm, using distinct sets of trajectories (consisting of 8138 and 15129 trajectories), collected in different stores and labeled by human reviewers. The calibrated models are then evaluated on trajectories held out of the calibration process both from the same store on which calibration was performed and from the other store. An analysis of the results shows that the algorithm can recognize customers' browsing activity when evaluated in an environment different from the one on which calibration was performed. We then use the model to analyze the customers' ``browsing patterns'' on a large set of trajectories and their relation to actual purchases in the stores. Finally, we discuss how shelf browsing information could be used for retail planning and in the domain of human-robot interaction scenarios.

Figures (6)

• Figure 1: Layouts of store $s1$ on the top and store $s2$ on the bottom. Blue polygons show the shelves with their interactive faces with a blue line which represents the face normal. Exits are shown in green and employee-only sections on yellow.
• Figure 2: Top view of a shopper that may be identified as browsing shelf $j$. $\lambda_j$ is the distance to the closest shelf, while $\lambda_s$ is the distance towards a farther shelf $s$.
• Figure 3: Same store $F_1$ score evaluation of the model. The $x$ axis shows the percentage of data used for calibration, while the $y$ axis shows the evaluation score on the remaining trajectories. The blue points correspond to 10 randomized choices of the calibration sets, the red points to the average, and the red bars to standard errors.
• Figure 4: Distribution of average visits per trip across shelves in store $s1$.
• Figure 5: Distribution of average visits per trip across shelves in store $s2$.