Embodied Navigation with Auxiliary Task of Action Description Prediction

TL;DR

This work tackles explainability in multimodal embodied navigation by introducing DescRL, an auxiliary-task framework that trains agents to describe their past and future actions during navigation. It employs a two-phase pipeline: Phase 1 pre-trains an ADGenerator to verbalize observations, and Phase 2 trains the navigation policy with an ADPredictor to predict action descriptions, optionally aided by knowledge distillation from vision-language models. Across ObjNav, VLN, and SAVNav, DescRL yields strong navigation performance, with the largest gains in the challenging SAVNav task, and enables analysis of failure modes through predicted descriptions. The approach reduces reliance on human-labeled data via VLM distillation and demonstrates that action-description auxiliary tasks can enhance both interpretability and, in many cases, navigation effectiveness.

Abstract

The field of multimodal robot navigation in indoor environments has garnered significant attention in recent years. However, as tasks and methods become more advanced, the action decision systems tend to become more complex and operate as black-boxes. For a reliable system, the ability to explain or describe its decisions is crucial; however, there tends to be a trade-off in that explainable systems can not outperform non-explainable systems in terms of performance. In this paper, we propose incorporating the task of describing actions in language into the reinforcement learning of navigation as an auxiliary task. Existing studies have found it difficult to incorporate describing actions into reinforcement learning due to the absence of ground-truth data. We address this issue by leveraging knowledge distillation from pre-trained description generation models, such as vision-language models. We comprehensively evaluate our approach across various navigation tasks, demonstrating that it can describe actions while attaining high navigation performance. Furthermore, it achieves state-of-the-art performance in the particularly challenging multimodal navigation task of semantic audio-visual navigation.

Figures (5)

• Figure 1: Overview of this study. The robot generates action descriptions of its past actions or future plans during navigation, such as moving toward a sounding object in the case of semantic audio-visual navigation, based on visual and auditory observations.
• Figure 2: Overview of DescRL applied to SAVi chen2020soundspaces, which is a method for semantic audio-visual navigation. The RL agent receives visual observation $V_t$, auditory observation $A_t$, posture $p_t$, and previous action $a_{t-1}$ at time $t$ and outputs the next action $a_t$ and the action description $\hat{D}_t$. $E^T$ and $E^P$ represent task embedding and positional encoding, respectively. Here, ADPredictor is trained as an auxiliary task to predict the output $D_t$ of the pre-trained ADGenerator.
• Figure 3: Navigation trajectories. Comparing the left and middle figures shows that DescRL reduces wasteful actions. The bottom right figure shows a failure case where the agent got close to the goal but failed to stop at the exact position.
• Figure 4: Qualitative evaluation of action descriptions. The agent generates the above action descriptions when it observes the RGBD image.
• Figure 5: Qualitative evaluation of generating action descrptions performance of ScaleVLN wang2023scaling w/ Past-DescRL on VLN. The agent observes the panoramic image at each step and must follow the given instructions. The red arrows represent the actions selected by the agent, and the action descriptions shown above each panorama image are ones generated by the agent. Areas marked in green are action descriptions that can be judged to be correct.