Foresight Social-aware Reinforcement Learning for Robot Navigation

TL;DR

A novel Foresight Social-aware Reinforcement Learning (FSRL) framework for mobile robots to achieve collision-free navigation that considers the current human-robot interaction to avoid an immediate collision, but also estimates upcoming social interactions to still keep distance in the future.

Abstract

When robots handle navigation tasks while avoiding collisions, they perform in crowded and complex environments not as good as in stable and homogeneous environments. This often results in a low success rate and poor efficiency. Therefore, we propose a novel Foresight Social-aware Reinforcement Learning (FSRL) framework for mobile robots to achieve collision-free navigation. Compared to previous learning-based methods, our approach is foresighted. It not only considers the current human-robot interaction to avoid an immediate collision, but also estimates upcoming social interactions to still keep distance in the future. Furthermore, an efficiency constraint is introduced in our approach that significantly reduces navigation time. Comparative experiments are performed to verify the effectiveness and efficiency of our proposed method under more realistic and challenging simulated environments.

Figures (8)

• Figure 1: In a realistic scenario, both standing (blue) and dynamic (grey) objects exist. Unlike oversimplified scenarios in previous methods where only dynamic objects exist, traps or blind spots will form more frequently and not disappear over time. These will trap the robot in the crowd more easily.
• Figure 2: Previous methods (brown) only detect the collision at time $t_2$ hence turn around sharply to avoid a collision, which is shortsighted. In contrast, our method (green) can forecast potential collision at time $t_1$ within $\Delta t$ (red) hence can take action in advance to avoid collision smoothly, which is foresighted. This is more important for nonholonomic robots as they cannot get out of traps easily.
• Figure 3: (a) An illustration of the effective range $r_{e}$ of the robot. The blue and grey circles respectively represent the standing and dynamic humans with radius ${r}^{i}$, where arrows indicate the moving direction. (b) By detecting potential collisions with standing objects in $\Delta T_{st}$, the robot can take action in advance to avoid collisions. This is achieved by the reward component $R_{st}$. (c) According to the comfortable distance $r_c$ within $\Delta T_{dy}$, the robot chooses to detour the crowd instead of going through it, which would annoy humans. This is achieved by the reward component $R_{dy}$.
• Figure 4: Simulation environments. Hollow circles are dynamic objects, whereas solid grey circles are static standing agents. (a) Env.1 has 10 dynamic objects. (b) Env.2 has 5 random standing objects and others are dynamic. (c) Env.3 has two group barriers of 2 and 3 standing objects and others are also dynamic. Compared to the previous environment, robots are more easily trapped and hard to detour in our latter two environments.
• Figure 5: Quantitative evaluation among ORCA10.1007/978-3-642-19457-3_1, LSTM-RL8593871, SARL 8794134 and FSRL(Ours) methods in three environments: (a) The three pictures on the left are with invisible setting; (b) The three pictures on the right are with visible setting.