PUGS: Perceptual Uncertainty for Grasp Selection in Underwater Environments

TL;DR

PUGS addresses autonomous underwater grasping under perceptual uncertainty by quantifying and propagating occupancy uncertainty from multi-view reconstruction into grasp selection. The framework builds a fused occupancy field from observational and pose-induced uncertainties, then uses a stochastic variational Gaussian process to obtain predictive occupancy uncertainty, which is fused via cubature integration. This predictive-occupancy uncertainty is subsequently used to weight grasp proposals from a baseline planner (TSGrasp), improving robustness in partial and noisy reconstructions while performing comparably on complete reconstructions. The approach is validated through simulations and real-world underwater experiments, demonstrating improved reliability and robustness for grasping under challenging perceptual conditions, with potential for real-time deployment with further optimization.

Abstract

When navigating and interacting in challenging environments where sensory information is imperfect and incomplete, robots must make decisions that account for these shortcomings. We propose a novel method for quantifying and representing such perceptual uncertainty in 3D reconstruction through occupancy uncertainty estimation. We develop a framework to incorporate it into grasp selection for autonomous manipulation in underwater environments. Instead of treating each measurement equally when deciding which location to grasp from, we present a framework that propagates uncertainty inherent in the multi-view reconstruction process into the grasp selection. We evaluate our method with both simulated and the real world data, showing that by accounting for uncertainty, the grasp selection becomes robust against partial and noisy measurements. Code will be made available at https://onurbagoren.github.io/PUGS/

Figures (4)

• Figure 1: Real-world underwater manipulation setup in the test tank at the University of Washington Applied Physics Laboratory (UW-APL). The test system includes a Reach Robotics Bravo 7 electric manipulator and Trisect subsea stereo sensor. The gripper in red shows the grasp pose from an out-of-the-box grasp selection model player_real-time_2023. The proposed method successfully leads to a more reliable grasping location, shown in green, by using perceptual uncertainty to weigh more favorable grasp locations. The visualizations are taken from evaluations of a real-world dataset and overlaid to match the photo showing the robot during regular operation.
• Figure 2: Overview of the proposed system. We use a SLAM pipeline to construct dense 3D reconstructions of objects of interest. PUGS quantifies the uncertainty inherent in the observation and pose estimation to construct an occupancy uncertainty representation. We use TSGrasp player_real-time_2023 as the baseline network to regress on grasp poses and confidences. The final grasp confidence and pose are determined by fusing the PUGS occupancy uncertainty output and the TSGrasp confidences.
• Figure 3: Selected results from simulation experiments with partial reconstructions of the kettlebell (first row) and noisy partial reconstructions of the coffee mug (second row). The first column shows the 3D object; the second column shows the grasps proposed by TSGrasp player_real-time_2023, and the third column shows the grasps after the adjustment made by PUGS. The first row shows the partially reconstructed kettlebell and the ability of PUGS to recover from an incorrect grasp pose prediction where TSGrasp attempts to grasp from the edge of the partial reconstruction and PUGS leads to a more reliable grasp. The second row shows a noisy reconstruction of the coffee mug, and a corrected grasp prediction by PUGS.
• Figure 4: Results from data collected from the test tank. The columns represent separate logs collected in the test tank. Each log contains images of partial views from different angles of the kettlebell. The first and second columns are from the same log, but the first column shows the pointcloud without the aggressive filtering. Here, we show a qualitative comparison of the proposed grasp poses from PUGS (green) and TSGrasp (red) player_real-time_2023. The reconstructions are colored by the weighted confidence output of PUGS and overlaid on a representative 3D model of the object for easy visualization. Higher confidence regions are colored in red, and lower confidence regions are colored in blue.