Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MAP-NBV: Multi-agent Prediction-guided Next-Best-View Planning for Active 3D Object Reconstruction

Harnaik Dhami, Vishnu D. Sharma, Pratap Tokekar

TL;DR

This paper designs a decentralized next-best-view approach that utilizes geometric measures over the predictions and jointly optimizes the information gain and control effort for efficient collaborative 3D reconstruction of the object.

Abstract

Next-Best View (NBV) planning is a long-standing problem of determining where to obtain the next best view of an object from, by a robot that is viewing the object. There are a number of methods for choosing NBV based on the observed part of the object. In this paper, we investigate how predicting the unobserved part helps with the efficiency of reconstructing the object. We present, Multi-Agent Prediction-Guided NBV (MAP-NBV), a decentralized coordination algorithm for active 3D reconstruction with multi-agent systems. Prediction-based approaches have shown great improvement in active perception tasks by learning the cues about structures in the environment from data. However, these methods primarily focus on single-agent systems. We design a decentralized next-best-view approach that utilizes geometric measures over the predictions and jointly optimizes the information gain and control effort for efficient collaborative 3D reconstruction of the object. Our method achieves 19% improvement over the non-predictive multi-agent approach in simulations using AirSim and ShapeNet. We make our code publicly available through our project website: http://raaslab.org/projects/MAPNBV/.

MAP-NBV: Multi-agent Prediction-guided Next-Best-View Planning for Active 3D Object Reconstruction

TL;DR

This paper designs a decentralized next-best-view approach that utilizes geometric measures over the predictions and jointly optimizes the information gain and control effort for efficient collaborative 3D reconstruction of the object.

Abstract

Next-Best View (NBV) planning is a long-standing problem of determining where to obtain the next best view of an object from, by a robot that is viewing the object. There are a number of methods for choosing NBV based on the observed part of the object. In this paper, we investigate how predicting the unobserved part helps with the efficiency of reconstructing the object. We present, Multi-Agent Prediction-Guided NBV (MAP-NBV), a decentralized coordination algorithm for active 3D reconstruction with multi-agent systems. Prediction-based approaches have shown great improvement in active perception tasks by learning the cues about structures in the environment from data. However, these methods primarily focus on single-agent systems. We design a decentralized next-best-view approach that utilizes geometric measures over the predictions and jointly optimizes the information gain and control effort for efficient collaborative 3D reconstruction of the object. Our method achieves 19% improvement over the non-predictive multi-agent approach in simulations using AirSim and ShapeNet. We make our code publicly available through our project website: http://raaslab.org/projects/MAPNBV/.
Paper Structure (13 sections, 1 equation, 7 figures, 1 table, 1 algorithm)

This paper contains 13 sections, 1 equation, 7 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Proposed Approach
  5. 3D Model Prediction (Line \ref{['algo:line:combined_observations']}-\ref{['algo:line:pc_completion']})
  6. Decentralized Coordination (Line \ref{['algo:line:cand_pose_generation']}-\ref{['algo:line:broadcast']})
  7. Experiments and Evaluation
  8. Setup
  9. Qualitative Example
  10. Can Point Cloud Prediction Improve Reconstruction?
  11. How does coordination affect reconstruction?
  12. Qualitative Real-World Experiment
  13. Conclusions

Figures (7)

  • Figure 1: MAP-NBV uses predictions to select better NBVs for a team of robots compared to the non-predictive baseline approach.
  • Figure 2: Algorithm Overview: Each robot runs the same algorithm including perception, prediction, and planning steps. The robots that communicate with each other can share observations and coordinate planning, whereas robots in isolation (e.g., Robot n) perform individual greedy planning.
  • Figure 3: Flight paths of the two robots during C-17 simulation.
  • Figure 4: Examples of the 5 simulation model classes.
  • Figure 5: MAP-NBV (CO(d)-CP(d)-Greedy) performs comparably to the optimal solution (CO(c)-CP(c)-Optimal; Section \ref{['sec:COCPdesc']}), and much better than the frontiers-baseline in AirSim experiments.
  • ...and 2 more figures