Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation

TL;DR

A systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures, which proves to enhance training stability and model performance and presents a deep reinforcement learning (DRL) framework for social robot navigation, which incorporates relational reasoning and trajectory prediction systematically.

Abstract

Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. While modeling pairwise relations has been widely studied in multi-agent interacting systems, the ability to capture larger-scale group-wise activities is limited. In this paper, we propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures, and we demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation. In addition to the edges between pairs of nodes (i.e., agents), we propose to infer hyperedges that adaptively connect multiple nodes to enable group-wise reasoning in an unsupervised manner. Our approach infers dynamically evolving relation graphs and hypergraphs to capture the evolution of relations, which the trajectory predictor employs to generate future states. Meanwhile, we propose to regularize the sharpness and sparsity of the learned relations and the smoothness of the relation evolution, which proves to enhance training stability and model performance. The proposed approach is validated on synthetic crowd simulations and real-world benchmark datasets. Experiments demonstrate that the approach infers reasonable relations and achieves state-of-the-art prediction performance. In addition, we present a deep reinforcement learning (DRL) framework for social robot navigation, which incorporates relational reasoning and trajectory prediction systematically. In a group-based crowd simulation, our method outperforms the strongest baseline by a significant margin in terms of safety, efficiency, and social compliance in dense, interactive scenarios. We also demonstrate the practical applicability of our method with real-world robot experiments. The code and videos can be found at https://relational-reasoning-nav.github.io/.

Figures (9)

• Figure 1: An illustrative example of social robot navigation in a dynamic, crowded scenario where the robot plans to reach its destination at the top left of the scene from its current location. Some pedestrians move individually while others behave similarly as a group. The relations between agents may evolve over time. The ellipses denote different groups of interacting agents that exhibit distinct group-wise relations, and the arrows indicate agents' motions. The robot needs to reach its destination safely and efficiently without colliding with pedestrians or intruding into group spaces.
• Figure 2: An overall diagram of our proposed pipeline, which consists of a trajectory prediction model and an RL agent for robot navigation. The prediction model takes in the historical observations of human motion and generates future trajectory hypotheses while inferring the underlying relations between humans. The RL agent decides the optimal robot actions based on the predicted trajectories and the current robot state. The inferred relations are used in designing the reward function. We decouple the training of the prediction model and the RL agent into two sequential phases. More details about the architectures and training strategy of the two components can be found in Sections \ref{['sec:evolvehypergraph']} and \ref{['sec:social-robot-navigation']}.
• Figure 3: A diagram of our prediction method within a single prediction period, which consists of an encoder for relational reasoning and a decoder to generate prediction hypotheses. Different colors of edges in the latent interaction graph and hyperedges (i.e., ellipses) in the latent interaction hypergraph indicate different relation types. Best viewed in color.
• Figure 4: A diagram of the social robot navigation framework, which consists of a trajectory predictor, a human-human/obstacle attention layer, a robot-human/obstacle attention layer, a value network, and a policy network. Best viewed in color.
• Figure 5: An illustrative visualization of two typical scenarios where the robot navigates in human crowds. (a) The robot is located outside the human groups' convex hulls, which complies with social norms. (b) The robot is located inside a human group's convex hull, which intrudes into human personal spaces and violates social norms. Best viewed in color.