Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MAction-SocialNav: Multi-Action Socially Compliant Navigation via Reasoning-enhanced Prompt Tuning

Zishuo Wang, Xinyu Zhang, Zhuonan Liu, Tomohito Kawabata, Daeun Song, Xuesu Xiao, Ling Xiao

TL;DR

MAction-SocialNav tackles action ambiguity in socially compliant navigation by learning to rank multiple plausible actions within a scene. It introduces a meta-cognitive prompt (MCP) to boost reasoning in a vision-language framework and curates a dedicated multi-action dataset with a grounded action space. The approach outperforms zero-shot large multimodal models in accuracy, safety alignment, and speed, while maintaining real-time performance. These contributions enable more robust, interpretable, and efficient social navigation in human-centered environments. The work also outlines practical paths for integrating high-level multi-action reasoning with low-level motion planning for real-world deployment.

Abstract

Socially compliant navigation requires robots to move safely and appropriately in human-centered environments by respecting social norms. However, social norms are often ambiguous, and in a single scenario, multiple actions may be equally acceptable. Most existing methods simplify this problem by assuming a single correct action, which limits their ability to handle real-world social uncertainty. In this work, we propose MAction-SocialNav, an efficient vision language model for socially compliant navigation that explicitly addresses action ambiguity, enabling generating multiple plausible actions within one scenario. To enhance the model's reasoning capability, we introduce a novel meta-cognitive prompt (MCP) method. Furthermore, to evaluate the proposed method, we curate a multi-action socially compliant navigation dataset that accounts for diverse conditions, including crowd density, indoor and outdoor environments, and dual human annotations. The dataset contains 789 samples, each with three-turn conversation, split into 710 training samples and 79 test samples through random selection. We also design five evaluation metrics to assess high-level decision precision, safety, and diversity. Extensive experiments demonstrate that the proposed MAction-SocialNav achieves strong social reasoning performance while maintaining high efficiency, highlighting its potential for real-world human robot navigation. Compared with zero-shot GPT-4o and Claude, our model achieves substantially higher decision quality (APG: 0.595 vs. 0.000/0.025) and safety alignment (ER: 0.264 vs. 0.642/0.668), while maintaining real-time efficiency (1.524 FPS, over 3x faster).

MAction-SocialNav: Multi-Action Socially Compliant Navigation via Reasoning-enhanced Prompt Tuning

TL;DR

MAction-SocialNav tackles action ambiguity in socially compliant navigation by learning to rank multiple plausible actions within a scene. It introduces a meta-cognitive prompt (MCP) to boost reasoning in a vision-language framework and curates a dedicated multi-action dataset with a grounded action space. The approach outperforms zero-shot large multimodal models in accuracy, safety alignment, and speed, while maintaining real-time performance. These contributions enable more robust, interpretable, and efficient social navigation in human-centered environments. The work also outlines practical paths for integrating high-level multi-action reasoning with low-level motion planning for real-world deployment.

Abstract

Socially compliant navigation requires robots to move safely and appropriately in human-centered environments by respecting social norms. However, social norms are often ambiguous, and in a single scenario, multiple actions may be equally acceptable. Most existing methods simplify this problem by assuming a single correct action, which limits their ability to handle real-world social uncertainty. In this work, we propose MAction-SocialNav, an efficient vision language model for socially compliant navigation that explicitly addresses action ambiguity, enabling generating multiple plausible actions within one scenario. To enhance the model's reasoning capability, we introduce a novel meta-cognitive prompt (MCP) method. Furthermore, to evaluate the proposed method, we curate a multi-action socially compliant navigation dataset that accounts for diverse conditions, including crowd density, indoor and outdoor environments, and dual human annotations. The dataset contains 789 samples, each with three-turn conversation, split into 710 training samples and 79 test samples through random selection. We also design five evaluation metrics to assess high-level decision precision, safety, and diversity. Extensive experiments demonstrate that the proposed MAction-SocialNav achieves strong social reasoning performance while maintaining high efficiency, highlighting its potential for real-world human robot navigation. Compared with zero-shot GPT-4o and Claude, our model achieves substantially higher decision quality (APG: 0.595 vs. 0.000/0.025) and safety alignment (ER: 0.264 vs. 0.642/0.668), while maintaining real-time efficiency (1.524 FPS, over 3x faster).
Paper Structure (17 sections, 7 equations, 6 figures, 5 tables)

This paper contains 17 sections, 7 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Social Robot Navigation
  4. VLM Reasoning.
  5. Method
  6. Overview
  7. Meta-Cognitive Prompt (MCP)
  8. Curated Multi-Action Dataset
  9. Proposed Evaluation Metrics
  10. Experiment
  11. Experimental Setup
  12. Main Results
  13. Comparison with Zero-Shot Large Multimodal Models
  14. Ablational Studies
  15. Performance Across Scenario Difficulty Levels
  16. ...and 2 more sections

Figures (6)

  • Figure 1: Task formulation and high-level concept of multi-action social navigation. Given multimodal observations, the agent is required to reason about the scene and generate multiple feasible navigation actions ranked by priority, rather than a single deterministic action. A meta-cognitive prompt (MCP) is proposed to stimulate structured reasoning and self-evaluation in the vision language model (VLM), enabling socially compliant and interpretable decision-making.
  • Figure 2: Overview of MAction-SocialNav Framework. We formulate socially compliant navigation as a multi-turn dialogue process. Given a scene observation $I$ and a designed MCP ($S_{\mathrm{MCP}}$) as the system prompt, the model sequentially performs perception and prediction through intermediate queries, and finally generates a ranked set of executable actions.
  • Figure 3: Multi-turn conversation dataset with action ranking. Each training sample consists of a visual observation paired with a multi-turn dialogue. All assistant responses are annotated by two human annotators. To support multi-action supervision, we introduce a hierarchical action ranking protocol. Specifically, we generate a ranked set of candidate actions based on a priority hierarchy: Feasibility$\rightarrow$ Social norms$\rightarrow$ Efficiency.
  • Figure 4: Constrained prompt used for querying GPT-4o and Claude.
  • Figure 5: Visual comparison with GPT-4o, Claude, MAction-SocialNav without MCP, and MAction-SocialNav with MCP. MAction-SocialNav with MCP predicts exactly the same set of feasible actions as the ground truth.
  • ...and 1 more figures