A short methodological review on social robot navigation benchmarking

TL;DR

This paper tackles the absence of standardized benchmarking in Social Robot Navigation (SocNav) by analyzing trends from 2020 to 2025 using IEEE Xplore as the data source. It systematically screens 130 studies down to 85 that include navigation algorithms and benchmarking elements, extracting metrics, baselines, human surveys, and interpretation practices. The study reveals a strong reliance on non-social metrics, a mix of social metrics with limited justification, and broad heterogeneity in benchmarking protocols and baselines. It highlights the need for clearer reporting of algorithms and configurations and argues for move toward more standardized, interpretable benchmarking to advance the field. The work provides actionable guidance to improve comparability and reproducibility in SocNav research.

Abstract

Social Robot Navigation is the skill that allows robots to move efficiently in human-populated environments while ensuring safety, comfort, and trust. Unlike other areas of research, the scientific community has not yet achieved an agreement on how Social Robot Navigation should be benchmarked. This is notably important, as the lack of a de facto standard to benchmark Social Robot Navigation can hinder the progress of the field and may lead to contradicting conclusions. Motivated by this gap, we contribute with a short review focused exclusively on benchmarking trends in the period from January 2020 to July 2025. Of the 130 papers identified by our search using IEEE Xplore, we analysed the 85 papers that met the criteria of the review. This review addresses the metrics used in the literature for benchmarking purposes, the algorithms employed in such benchmarks, the use of human surveys for benchmarking, and how conclusions are drawn from the benchmarking results, when applicable.

Figures (3)

• Figure 1: PRISMA flow diagram of the process followed.
• Figure 2: On the left, an overview of the overall screening decisions. On the right, the distribution of the rationale for the application of the second exclusion criterion.
• Figure 3: Sankey diagram showing, left to right, a categorisation of metrics in social vs. non-social, the general variables measured by the metrics, the metrics found in the review, the works reviewed in this paper, and the baselines they used.