Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Socially Integrated Navigation: A Social Acting Robot with Deep Reinforcement Learning

Daniel Flögel, Lars Fischer, Thomas Rudolf, Tobias Schürmann, Sören Hohmann

TL;DR

This work addresses socially navigating mobile robots in crowded environments by introducing a socially integrated navigation paradigm in which the robot's social behavior emerges from interactions with humans rather than being fixed a priori. The approach uses deep reinforcement learning with an adaptive reward structure that aggregates social cues from nearby humans within a social integration radius, and trains in environments where humans themselves follow social norms. A taxonomy distinguishing social collision avoidance, socially aware, and socially integrated navigation is proposed, along with a practical PPO-based training framework that leverages an LSTM feature extractor and velocity-based incentives to avoid imitation. Evaluation across passing and circle-crossing scenarios shows that socially integrated navigation improves ego navigation performance while reducing negative impact on humans, indicating scalable, socially acceptable robot behavior in crowded settings. Real-world validation is identified as future work to confirm practicality outside simulation.

Abstract

Mobile robots are being used on a large scale in various crowded situations and become part of our society. The socially acceptable navigation behavior of a mobile robot with individual human consideration is an essential requirement for scalable applications and human acceptance. Deep Reinforcement Learning (DRL) approaches are recently used to learn a robot's navigation policy and to model the complex interactions between robots and humans. We propose to divide existing DRL-based navigation approaches based on the robot's exhibited social behavior and distinguish between social collision avoidance with a lack of social behavior and socially aware approaches with explicit predefined social behavior. In addition, we propose a novel socially integrated navigation approach where the robot's social behavior is adaptive and emerges from the interaction with humans. The formulation of our approach is derived from a sociological definition, which states that social acting is oriented toward the acting of others. The DRL policy is trained in an environment where other agents interact socially integrated and reward the robot's behavior individually. The simulation results indicate that the proposed socially integrated navigation approach outperforms a socially aware approach in terms of ego navigation performance while significantly reducing the negative impact on all agents within the environment.

Socially Integrated Navigation: A Social Acting Robot with Deep Reinforcement Learning

TL;DR

This work addresses socially navigating mobile robots in crowded environments by introducing a socially integrated navigation paradigm in which the robot's social behavior emerges from interactions with humans rather than being fixed a priori. The approach uses deep reinforcement learning with an adaptive reward structure that aggregates social cues from nearby humans within a social integration radius, and trains in environments where humans themselves follow social norms. A taxonomy distinguishing social collision avoidance, socially aware, and socially integrated navigation is proposed, along with a practical PPO-based training framework that leverages an LSTM feature extractor and velocity-based incentives to avoid imitation. Evaluation across passing and circle-crossing scenarios shows that socially integrated navigation improves ego navigation performance while reducing negative impact on humans, indicating scalable, socially acceptable robot behavior in crowded settings. Real-world validation is identified as future work to confirm practicality outside simulation.

Abstract

Mobile robots are being used on a large scale in various crowded situations and become part of our society. The socially acceptable navigation behavior of a mobile robot with individual human consideration is an essential requirement for scalable applications and human acceptance. Deep Reinforcement Learning (DRL) approaches are recently used to learn a robot's navigation policy and to model the complex interactions between robots and humans. We propose to divide existing DRL-based navigation approaches based on the robot's exhibited social behavior and distinguish between social collision avoidance with a lack of social behavior and socially aware approaches with explicit predefined social behavior. In addition, we propose a novel socially integrated navigation approach where the robot's social behavior is adaptive and emerges from the interaction with humans. The formulation of our approach is derived from a sociological definition, which states that social acting is oriented toward the acting of others. The DRL policy is trained in an environment where other agents interact socially integrated and reward the robot's behavior individually. The simulation results indicate that the proposed socially integrated navigation approach outperforms a socially aware approach in terms of ego navigation performance while significantly reducing the negative impact on all agents within the environment.
Paper Structure (19 sections, 9 equations, 6 figures, 2 tables)

This paper contains 19 sections, 9 equations, 6 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Social Navigation
  3. Social Behavior in Crowded Environments
  4. Proposed Categorization for DRL Navigation Approaches
  5. Social Collision Avoidance
  6. Socially Aware Navigation
  7. Socially Integrated Navigation
  8. Preliminaries
  9. Problem Formulation
  10. Approach
  11. Adapting Social Behavior
  12. Training a Socially Integrated Agent
  13. Evaluation
  14. Experimental Setup
  15. Results
  16. ...and 4 more sections

Figures (6)

  • Figure 1: The proposed socially integrated navigation approach is adaptive to human behavior and preferences. It is based on a perspective change where the social behavior of the robot arises from its interaction with humans and is not predefined. The drl policy has learned from the interaction behavior to consider the unknown personal space of each human.
  • Figure 2: We propose to distinguish between different drl-based navigation approaches among pedestrians based on robot's exhibited social interactive behavior. Social collision avoidance approaches consider only collision-free navigation. In socially aware approaches, a predetermined social behavior is projected onto all humans in the environment. Adaptive social behavior is considered in our socially integrated approaches, where humans' individual social behavior is considered, and the robot's behavior arises from an interaction with humans.
  • Figure 3: Proposed distributed rewards in the environment. The robot has only one navigation reward and receives the reward for the social behavior of humans within the social integration radius $r_{\text{SI}}$.
  • Figure 4: Proposed perspective of the various agents among each other in the environment. The gray-shaded personal areas are only visible to the respective agent. Humans act in the environment to maintain their personal space and that of others.
  • Figure 5: Proposed architecture with shared features extractor and separated policy and critic networks. The last hidden state of the LSTM is used as input for fully connected layers.
  • ...and 1 more figures