Marker-free Human Gait Analysis using a Smart Edge Sensor System

TL;DR

A novel markerless approach for gait analysis using a multi-camera setup with smart edge sensors to estimate 3D body poses without fiducial markers is introduced and a Siamese embedding network with triplet loss calculation is proposed to identify individuals by their gait pattern.

Abstract

The human gait is a complex interplay between the neuronal and the muscular systems, reflecting an individual's neurological and physiological condition. This makes gait analysis a valuable tool for biomechanics and medical experts. Traditional observational gait analysis is cost-effective but lacks reliability and accuracy, while instrumented gait analysis, particularly using marker-based optical systems, provides accurate data but is expensive and time-consuming. In this paper, we introduce a novel markerless approach for gait analysis using a multi-camera setup with smart edge sensors to estimate 3D body poses without fiducial markers. We propose a Siamese embedding network with triplet loss calculation to identify individuals by their gait pattern. This network effectively maps gait sequences to an embedding space that enables clustering sequences from the same individual or activity closely together while separating those of different ones. Our results demonstrate the potential of the proposed system for efficient automated gait analysis in diverse real-world environments, facilitating a wide range of applications.

Figures (7)

• Figure 1: Sample movement sequence (b) captured with smart edge sensors (a) consisting of a Nvidia Jetson Orin compute board and an Intel RealSense RGB-D camera. (c) t-SNE visualization of activities from Human 3.6M dataset based on the learned gait embedding.
• Figure 2: Example of the 3D skeleton model with $J=19$ joints (a), coded by color. The red circle marks the root joint used for normalization (b); (c) RGB-representation of one $(J,T,3)$ input tensor with $T=30$ frames.
• Figure 3: Gait embedding network with semi-hard mining used for the training process. The input is a data batch of $N$ tensors that hold the 3D position of $J$ joints over sequences of $T$ frames. The input is processed by the Siamese network which computes $D$-dimensional embedding vectors. Every embedding vector is matched with positive samples of the same person and negative samples of a different one to form triplets. The negative sample is selected by searching for embeddings close to the anchor vector to make the learning process more efficient. With the selected triplets, the loss is calculated and backpropagated through the network. After training, the network is able to map the same person close by and different ones apart.
• Figure 4: t-SNE projections of the validation data. Each dot represents a validation sequence with Participant ID coded by color. (a) raw input data, (b) embeddings computed with final model after $10,000$ training epochs.
• Figure 5: t-SNE plot for validation of the model with gait data of two unknown participants 21 (red) and 22 (orange), highlighted with red circles.