Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shopformer: Transformer-Based Framework for Detecting Shoplifting via Human Pose

Narges Rashvand, Ghazal Alinezhad Noghre, Armin Danesh Pazho, Babak Rahimi Ardabili, Hamed Tabkhi

TL;DR

Shopformer shifts shoplifting detection from pixel-based video analysis to pose-sequence modeling by combining a Graph Convolutional Autoencoder–based tokenizer with a transformer encoder-decoder. The two-stage training first learns rich spatio-temporal pose embeddings, which are then tokenized for efficient sequence modeling; reconstruction error on normal pose sequences serves as the anomaly score. On PoseLift, Shopformer achieves state-of-the-art AUC-ROC among pose-based detectors and demonstrates a favorable balance between discrimination and efficiency, with ablations showing that two tokens plus a 144-channel embedding optimize performance. The approach offers privacy-preserving, real-time surveillance potential for retail environments, highlighting the viability of pose-centric transformer architectures for behavior-specific anomaly detection.

Abstract

Shoplifting remains a costly issue for the retail sector, but traditional surveillance systems, which are mostly based on human monitoring, are still largely ineffective, with only about 2% of shoplifters being arrested. Existing AI-based approaches rely on pixel-level video analysis which raises privacy concerns, is sensitive to environmental variations, and demands significant computational resources. To address these limitations, we introduce Shopformer, a novel transformer-based model that detects shoplifting by analyzing pose sequences rather than raw video. We propose a custom tokenization strategy that converts pose sequences into compact embeddings for efficient transformer processing. To the best of our knowledge, this is the first pose-sequence-based transformer model for shoplifting detection. Evaluated on real-world pose data, our method outperforms state-of-the-art anomaly detection models, offering a privacy-preserving, and scalable solution for real-time retail surveillance. The code base for this work is available at https://github.com/TeCSAR-UNCC/Shopformer.

Shopformer: Transformer-Based Framework for Detecting Shoplifting via Human Pose

TL;DR

Shopformer shifts shoplifting detection from pixel-based video analysis to pose-sequence modeling by combining a Graph Convolutional Autoencoder–based tokenizer with a transformer encoder-decoder. The two-stage training first learns rich spatio-temporal pose embeddings, which are then tokenized for efficient sequence modeling; reconstruction error on normal pose sequences serves as the anomaly score. On PoseLift, Shopformer achieves state-of-the-art AUC-ROC among pose-based detectors and demonstrates a favorable balance between discrimination and efficiency, with ablations showing that two tokens plus a 144-channel embedding optimize performance. The approach offers privacy-preserving, real-time surveillance potential for retail environments, highlighting the viability of pose-centric transformer architectures for behavior-specific anomaly detection.

Abstract

Shoplifting remains a costly issue for the retail sector, but traditional surveillance systems, which are mostly based on human monitoring, are still largely ineffective, with only about 2% of shoplifters being arrested. Existing AI-based approaches rely on pixel-level video analysis which raises privacy concerns, is sensitive to environmental variations, and demands significant computational resources. To address these limitations, we introduce Shopformer, a novel transformer-based model that detects shoplifting by analyzing pose sequences rather than raw video. We propose a custom tokenization strategy that converts pose sequences into compact embeddings for efficient transformer processing. To the best of our knowledge, this is the first pose-sequence-based transformer model for shoplifting detection. Evaluated on real-world pose data, our method outperforms state-of-the-art anomaly detection models, offering a privacy-preserving, and scalable solution for real-time retail surveillance. The code base for this work is available at https://github.com/TeCSAR-UNCC/Shopformer.
Paper Structure (17 sections, 5 equations, 4 figures, 5 tables)

This paper contains 17 sections, 5 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Shopformer
  4. Tokenizer Module Based on GCAE Encoder
  5. Transformer Module
  6. Experimental Setup
  7. Dataset
  8. Metrics
  9. AUC-ROC
  10. AUC-PR
  11. Equal Error Rate (EER)
  12. Training and evaluation Strategy
  13. Token number selection: Finding the Optimal Trade-off
  14. Results
  15. Conclusion
  16. ...and 2 more sections

Figures (4)

  • Figure 1: This map shows the revenue lost to retail shoplifting across U.S. states (in billions), highlighting the geographical distribution of shoplifting-related losses, and identifying the states experiencing the highest prevalence of retail shoplifting nationwide capitalone2025shoplifting.
  • Figure 2: Represents the status of facial recognition technology (FRT) laws in US states. Dark red states have strong limitations, while light red states have lighter limitations on the use of FRT. States without enacted state-level laws are shown in gray. * Although California lacks state-level FRT regulations, some of its counties impose strict restrictions.
  • Figure 3: Overview of the Shopformer architecture. The framework operates in two stages: (1) a Graph Convolutional Autoencoder is first trained on pose sequences to learn rich spatio-temporal representations; (2) the pretrained encoder is then repurposed as a tokenizer module, generating compact tokens from input pose data. These tokens are passed through a transformer encoder-decoder module, which reconstructs the input sequence. The reconstruction error (MSE loss) is used to compute the normality score for shoplifting detection.
  • Figure 4: Comparison of AUC-ROC, AUC-PR, and EER across different token counts, highlighting 2 tokens as the optimal choice with the highest AUC-ROC (0.6915) and lowest EER (0.3819).