Using joint angles based on the international biomechanical standards for human action recognition and related tasks

TL;DR

This work addresses the need for interpretable, robust representations of human motion for machine learning by translating skeletal keypoint data into joint angles aligned with International Society of Biomechanics (ISB) standards. It defines local joint coordinate systems (JCS) at each joint and a bijective, viewpoint- and subject-invariant angle representation that enables pose recovery given bone lengths. Through experiments on MHAD, Skeletics152, and NTU RGB+D, joint angles show competitive performance with keypoints, with clear advantages in rotation robustness and subtle action discrimination, and they prove complementary when ensembled with keypoint models. A Python package to compute ISB-based joint angles from common keypoint sets is released to promote adoption in medical and sports science contexts where interpretability and human expert dialogue are crucial.

Abstract

Keypoint data has received a considerable amount of attention in machine learning for tasks like action detection and recognition. However, human experts in movement such as doctors, physiotherapists, sports scientists and coaches use a notion of joint angles standardised by the International Society of Biomechanics to precisely and efficiently communicate static body poses and movements. In this paper, we introduce the basic biomechanical notions and show how they can be used to convert common keypoint data into joint angles that uniquely describe the given pose and have various desirable mathematical properties, such as independence of both the camera viewpoint and the person performing the action. We experimentally demonstrate that the joint angle representation of keypoint data is suitable for machine learning applications and can in some cases bring an immediate performance gain. The use of joint angles as a human meaningful representation of kinematic data is in particular promising for applications where interpretability and dialog with human experts is important, such as many sports and medical applications. To facilitate further research in this direction, we will release a python package to convert keypoint data into joint angles as outlined in this paper.

Figures (8)

• Figure 1: The three planes of motion: sagittal (xy, red), frontal (yz, blue) and transverse (xz, green).
• Figure 2: A visualisation of all JCS and the computation of shoulder & neck angles.
• Figure 3: Performance comparison of joint angles and keypoints per dataset. Each small symbol represents one hyperparameter configuration with light dots for angles and dark crosses for keypoints, and similar colours for each model class. Large circles and squares represent the average performance of a given model class on joint angles and keypoints respectively.
• Figure 4: Performance comparison of joint angles and keypoints per model class. Each small symbol represents one hyperparameter configuration with light dots for angles and dark crosses for keypoints, and similar colours for each dataset. Large circles and squares represent the average performance on a given dataset on joint angles and keypoints respectively.
• Figure 5: Performance comparison of transformers by dataset size.