A Decade of Action Quality Assessment: Largest Systematic Survey of Trends, Challenges, and Future Directions

TL;DR

This paper provides the most comprehensive synthesis to date of Action Quality Assessment (AQA), surveying over 200 papers and 26 datasets using the PRISMA framework up to December 2024. It maps the field across problem definitions, metrics, and a wide range of datasets, then distills seven key methodological trends from fundamental research to self-supervised and explainable AQA. It highlights the convergence toward fine-grained, multimodal, generalizable, and interpretable models, including uncertainty-aware and contrastive approaches, and notes GAIA as a landmark AI-generated dataset. The authors discuss persistent challenges—dataset scale, action diversity, annotation quality, model interpretability, and robustness to missing modalities—and propose directions such as large-scale multi-action real-world datasets and AI-generated data to accelerate progress.

Abstract

Action Quality Assessment (AQA) -- the ability to quantify the quality of human motion, actions, or skill levels and provide feedback -- has far-reaching implications in areas such as low-cost physiotherapy, sports training, and workforce development. As such, it has become a critical field in computer vision & video understanding over the past decade. Significant progress has been made in AQA methodologies, datasets, & applications, yet a pressing need remains for a comprehensive synthesis of this rapidly evolving field. In this paper, we present a thorough survey of the AQA landscape, systematically reviewing over 200 research papers using the preferred reporting items for systematic reviews & meta-analyses (PRISMA) framework. We begin by covering foundational concepts & definitions, then move to general frameworks & performance metrics, & finally discuss the latest advances in methodologies & datasets. This survey provides a detailed analysis of research trends, performance comparisons, challenges, & future directions. Through this work, we aim to offer a valuable resource for both newcomers & experienced researchers, promoting further exploration & progress in AQA. Data are available at https://haoyin116.github.io/Survey_of_AQA/

Figures (20)

• Figure 1: AQA model plays the role of referee to evaluate how well actions are performed and score the skill level of performers.
• Figure 2: The number of papers related to AQA over the past decade.
• Figure 3: Structure of the overall survey. Zoom in for the best view.
• Figure 4: Execution quality vs difficulty level. A professional cliff diver is shown doing a backflip in (a). Notice how straight his legs are, how crisp his overall form is, and how graceful his execution is, compared to an amateur diving in (b). Amateur's legs are bent at the knees, as rotating with bent legs would result in faster rotation (lesser effort on the diver's part). It generally requires a lot of training and practice to be able to attain and maintain a great form like (a). Now, we compare two amateurs (b) and (c). Even though both have bad forms, doing a backflip, as in (b), is more difficult (requires more effort and skills) than simply falling into the lake like (c).
• Figure 5: Examples of elements that matter in AQA. Zoom in for the best view.