IPPON: Common Sense Guided Informative Path Planning for Object Goal Navigation

TL;DR

Although this planner follows a zero-shot approach, it achieves state-of-the-art performance as measured by the Success weighted by Path Length (SPL) and Soft SPL in the Habitat ObjectNav Challenge 2023, outperforming other works by more than 20%.

Abstract

Navigating efficiently to an object in an unexplored environment is a critical skill for general-purpose intelligent robots. Recent approaches to this object goal navigation problem have embraced a modular strategy, integrating classical exploration algorithms-notably frontier exploration-with a learned semantic mapping/exploration module. This paper introduces a novel informative path planning and 3D object probability mapping approach. The mapping module computes the probability of the object of interest through semantic segmentation and a Bayes filter. Additionally, it stores probabilities for common objects, which semantically guides the exploration based on common sense priors from a large language model. The planner terminates when the current viewpoint captures enough voxels identified with high confidence as the object of interest. Although our planner follows a zero-shot approach, it achieves state-of-the-art performance as measured by the Success weighted by Path Length (SPL) and Soft SPL in the Habitat ObjectNav Challenge 2023, outperforming other works by more than 20%. Furthermore, we validate its effectiveness on real robots. Project webpage: https://ippon-paper.github.io/

Figures (5)

• Figure 1: The pipeline of IPPON consists of three main components: 3D object probability mapping using the Bayes Filter, common sense reasoning that provides a proximity map for the OOI, and informative path planning based on the probability map combined with the proximity map. IPPON uses Voxblox oleynikova2017voxblox to compute the Euclidean signed distance field (ESDF) online for traversability estimation. Notice that we are showing the map and plan from the previous step, so the detected bed is not yet reflected on the probability map. We color-code the nodes in IPP: terminating nodes in white, minimally-exploring nodes in black, and the rest based on their gain -- from highest (red) to lowest (blue). In this example of finding a bed, the robot initially detects a chest of drawers in the top right corner. Due to its close proximity, the planner subsequently focuses on exploring the nearby area (red nodes), leading it to terminate in front of the bed. The final map and plan are shown in the black-outlined image.
• Figure 2: Calculation of $p_{img}$ for the scene in Fig. \ref{['fig:ippon-pipeline']}: the orange node near a chest of drawers has a high probability ($p_{near}$) of locating a bed, while the blue node near the counter has a low probability ($p_{far}$).
• Figure 3: The local SBP can find a path in a narrow "L"-shaped corridor when a straight-line connection fails.
• Figure 4: The robot finds eight different objects in succession: a toy elephant, a screwdriver, a microwave oven, a coffee machine, a plant, a TV, a chair, and a sofa. Notice that the eight trajectories are connected. This is because we send the next target immediately when the robot locates the current OOI and the robot can reuse the probability map and ESDF. The OOI in the final RGB image is shown (cropped) and outlined in a color corresponding to its trajectory.
• Figure 5: The robot navigates a challenging environment to locate a microwave oven, requiring it to traverse at least 30 meters and make a minimum of two turns. We visualize the camera images in gray rectangular callout.