Deep Learning based approach to detect Customer Age, Gender and Expression in Surveillance Video

TL;DR

This paper tackles the problem of deriving customer demographics and expressions from surveillance video to support in-store marketing analytics. It presents a modular pipeline that detects faces using Haar Cascade, then estimates age and gender with a Wide Residual Network (WRN-16-8) on $64\times 64$ face crops, and finally recognizes expressions with a mini Xception model. On a real-world garment-store dataset, the approach achieves $82.9\%$ gender accuracy and $70.8\%$ age-range accuracy, with expressions reasonably aligned to human judgments. The method demonstrates the practicality of performing demographic and affective analytics in challenging, low-resolution surveillance settings, while highlighting avenues for extension to more diverse data and environments.

Abstract

In the current information era, customer analytics play a key role in the success of any business. Since customer demographics primarily dictate their preferences, identification and utilization of age & gender information of customers in sales forecasting, may maximize retail sales. In this work, we propose a computer vision based approach to age and gender prediction in surveillance video. The proposed approach leverage the effectiveness of Wide Residual Networks and Xception deep learning models to predict age and gender demographics of the consumers. The proposed approach is designed to work with raw video captured in a typical CCTV video surveillance system. The effectiveness of the proposed approach is evaluated on real-life garment store surveillance video, which is captured by low resolution camera, under non-uniform illumination, with occlusions due to crowding, and environmental noise. The system can also detect customer facial expressions during purchase in addition to demographics, that can be utilized to devise effective marketing strategies for their customer base, to maximize sales.

Figures (5)

• Figure 1: Workflow of the proposed Framework
• Figure 2: WRN-16-8 model architecture
• Figure 3: mini Xception model architecture
• Figure 4: Visualization of results for video
• Figure 5: Sample video in the dataset