Metrics vs Surveys: Can Quantitative Measures Replace Human Surveys in Social Robot Navigation? A Correlation Analysis

TL;DR

This work addresses whether quantitative social navigation metrics can substitute for costly human surveys by analyzing correlations between $QM$ and human-centric judgments. A correlation-based framework combines $K$-means clustering and nonparametric statistics (Spearman's $\rho$ and Kendall's $\tau$) to identify a minimal $QM$ subset that mirrors human perceptions across eight real-world scenarios and 24 experiments with 70 participants. The authors provide a dataset of robot and pedestrian trajectories, a survey instrument, and show that an optimal QM subset (e.g., time-to-goal, average robot speed, proxemics in intimate/social zones, and average minimum distance to people) reproduces HM trends better than the full $QM$ set, reducing reliance on surveys. However, certain advanced and mixed scenarios still pose evaluation challenges, indicating the need for new metrics and learning-based methods to fully capture subjective aspects like comfort and legibility.

Abstract

Social, also called human-aware, navigation is a key challenge for the integration of mobile robots into human environments. The evaluation of such systems is complex, as factors such as comfort, safety, and legibility must be considered. Human-centered assessments, typically conducted through surveys, provide reliable insights but are costly, resource-intensive, and difficult to reproduce or compare across systems. Alternatively, numerical social navigation metrics are easy to compute and facilitate comparisons, yet the community lacks consensus on a standard set of metrics. This work explores the relationship between numerical metrics and human-centered evaluations to identify potential correlations. If specific quantitative measures align with human perceptions, they could serve as standardized evaluation tools, reducing the dependency on surveys. Our results indicate that while current metrics capture some aspects of robot navigation behavior, important subjective factors remain insufficiently represented and new metrics are necessary.

Figures (6)

• Figure 1: The proposed correlation-based framework to infer human metrics (HM) for social navigation experiment using an optimal set of representative quantitative metrics QM*.
• Figure 2: The laboratory where experiments were performed using a Jackal robot and a VICON tracking system to collect trajectories.
• Figure 3: Maps used for the experiments and trajectories of robot and humans in four distinct scenarios: Passing, Crossing 3, Narrow turn and Mixed. Velocity scale is indicated with the colorbar at the side, while temporal evolution is expressed with transparency. Other scenarios differ in human motion, as explained in Section \ref{['subsec:social_scenarios']}.
• Figure 4: Histogram of the cumulative ARI. The QM metrics with the highest cumulative ARI result are more relevant for clustering experiments as done using the HM feature set.
• Figure 5: Heatmap of consistent correlations between single QM and HM metrics, according to both Kendall and Spearman analysis.