Bench-NPIN: Benchmarking Non-prehensile Interactive Navigation

TL;DR

Bench-NPIN addresses the need for standardized evaluation in non-prehensile interactive navigation by introducing four configurable simulated environments (Maze, Ship-Ice, Box-Delivery, Area-Clearing) and two task classes (navigation-centric and manipulation-centric). It combines Gymnasium-compatible environments built on Pymunk with an extensible policy interface and a mix of RL and task-specific baselines, demonstrated through comprehensive cross-task evaluations. The benchmark introduces novel metrics for interactive navigation, including $E_{ ext{nav}} = \mathds{1}_{\text{success}}\frac{l_0^*}{l_0}$ and $I_{ ext{nav}} = \frac{m_0 l_0}{\sum_{i=0}^K m_i l_i}$, plus manipulation metrics $S_{ ext{manip}}$, $E_{ ext{manip}}$, and $I_{ ext{manip}}$, to quantify efficiency and interaction effort. By providing an open-source, modular toolkit with baseline policies and detailed evaluations, Bench-NPIN enables reproducible comparisons and paves the way for future extensions to 3D environments, demonstration data, and real-robot validation.

Abstract

Mobile robots are increasingly deployed in unstructured environments where obstacles and objects are movable. Navigation in such environments is known as interactive navigation, where task completion requires not only avoiding obstacles but also strategic interactions with movable objects. Non-prehensile interactive navigation focuses on non-grasping interaction strategies, such as pushing, rather than relying on prehensile manipulation. Despite a growing body of research in this field, most solutions are evaluated using case-specific setups, limiting reproducibility and cross-comparison. In this paper, we present Bench-NPIN, the first comprehensive benchmark for non-prehensile interactive navigation. Bench-NPIN includes multiple components: 1) a comprehensive range of simulated environments for non-prehensile interactive navigation tasks, including navigating a maze with movable obstacles, autonomous ship navigation in icy waters, box delivery, and area clearing, each with varying levels of complexity; 2) a set of evaluation metrics that capture unique aspects of interactive navigation, such as efficiency, interaction effort, and partial task completion; and 3) demonstrations using Bench-NPIN to evaluate example implementations of established baselines across environments. Bench-NPIN is an open-source Python library with a modular design. The code, documentation, and trained models can be found at https://github.com/IvanIZ/BenchNPIN.

Figures (10)

• Figure 1: Illustration of non-prehensile interactive navigation environments with configurable complexity levels. Upper-left shows the Maze environment, the mobile robot must reach the goal point among movable obstacles. Lower-left shows the Ship-Ice environment where an autonomous ship must navigate through ice-covered waters. Upper-right shows Box-Delivery environment, where the robot's objective is to push all boxes into the green receptacle. Lower-right is the Area-Clearing environment, where a robot needs to remove all boxes from within a clearance area outlined by the green rectangle.
• Figure 2: The Maze environment (left) and an example egocentric observation for Maze (right).
• Figure 3: The Area-Clearing environment (left) and example observation for Area-Clearing (right).
• Figure 4: An illustration of three teleoperated paths and the corresponding performance. While the shortest path (red) has the highest efficiency score $E_{\text{nav}} = 0.86$, the excessive collisions with the movable obstacles significantly degrade the effort score $I_{\text{nav}} = 0.56$. In contrast, while the longest path (blue) is collision-free $I_{\text{nav}} = 1.00$, efficiency is largely compromised $E_{\text{nav}} = 0.54$. A balanced path (yellow) potentially offers the best trade-off.
• Figure 5: Three teleoperated paths for area clearing, with each path having a task success score $S_\text{manip} = 2/3$. While path 1 (red) has the highest efficiency score, its effort score is low as it moves a box longer than its shortest distance to the goal (dotted lines of length $l_i^*$). In contrast, path 3 (blue) achieves the best effort score by pushing as little as possible, but results in a long path and a low efficiency score. Path 2 (yellow) potentially offers the best trade-off.