WeVibe: Weight Change Estimation Through Audio-Induced Shelf Vibrations In Autonomous Stores

TL;DR

WeVibe addresses the challenge of estimating weight changes on autonomous-store shelves with minimal sensor hardware by using audio-induced shelf vibrations as an active sensing mechanism. A physics-informed, largely linear relationship between shelf vibration spectra and item weight is derived and validated across multiple shelf locations, enabling data-efficient learning with per-location models. Empirical and theoretical results show mean absolute errors around 38–48 g using a single sensor and a small fraction of training data, including real-item layouts such as protein bars, indicating practical viability and scalability for deployment in real stores.

Abstract

Weight change estimation is crucial in various applications, particularly for detecting pick-up and put-back actions when people interact with the shelf while shopping in autonomous stores. Moreover, accurate weight change estimation allows autonomous stores to automatically identify items being picked up or put back, ensuring precise cost estimation. However, the conventional approach of estimating weight changes requires specialized weight-sensing shelves, which are densely deployed weight scales, incurring intensive sensor consumption and high costs. Prior works explored the vibration-based weight sensing method, but they failed when the location of weight change varies. In response to these limitations, we made the following contributions: (1) We propose WeVibe, a first item weight change estimation system through active shelf vibration sensing. The main intuition of the system is that the weight placed on the shelf influences the dynamic vibration response of the shelf, thus altering the shelf vibration patterns. (2) We model a physics-informed relationship between the shelf vibration response and item weight across multiple locations on the shelf based on structural dynamics theory. This relationship is linear and allows easy training of a weight estimation model at a new location without heavy data collection. (3) We evaluate our system on a gondola shelf organized as the real-store settings. WeVibe achieved a mean absolute error down to 38.07g and a standard deviation of 31.2g with one sensor and 10% samples from three weight classes on estimating weight change from 0g to 450g, which can be leveraged for differentiating items with more than 100g differences.

Figures (8)

• Figure 1: WeVibe system diagram comprises (a) An audio-induced vibration module designed for the active vibration sensing environment. (b) The vibration sensing module that takes shelf vibration response when there are different weights on the shelf. (c) The weight change estimation module that leverages physics-informed knowledge to extract features and develop a learning model for weight estimation. The weight change is achieved by calculating the difference between the two weight estimation results.
• Figure 2: (a) The plot shows the frequency spectrums of different weights of a single item at two locations. For the same location, some weight-sensitive frequencies increase or decrease while the weight of the item increases, as indicated by the red circle. Furthermore, when the item changes location, the overall frequency spectrum has a more significant change, and the weight-sensitive frequencies also shift. (b) We Further plot the result of linear regression on the highlighted frequencies and item weight at both locations. Even though some points deviate from the fitted line, the visualization gives rise to the assumption of linearity.
• Figure 3: (a) shows the store gondola and active vibration sensing setup with a speaker next to the gondola. The top right signal clip shows an example of our given vibration signal. (b) gives a more detailed view of the item location and sensor location. The different weight classes are taken by changing the amount of water in the 1L water bottle. (c) provides an overview of our vibration sensing module.
• Figure 4: The simplified shelf model for developing the theoretical model.
• Figure 5: The WeVibe system evaluation. (a) The comparison between WeVibe and the other two methods: Using one linear model for all locations and using the non-linear model for each location while using 10% of all weight classes in training and one vibration sensor. WeVibe outperforms both approaches with a significant improvement. (b)&(c) The weight change estimation result of WeVibe, taking 10% of 3 weight classes in training and one vibration sensor. The result suggests that WeVibe can almost certainly distinguish weight changes bigger than 100g, which can be utilized to detect whether a can of chips or a bottle of water is taken or put back.