Integrating One-Shot View Planning with a Single Next-Best View via Long-Tail Multiview Sampling

TL;DR

A novel combined pipeline is introduced that incorporates a single NBV before activating the proposed multiview-activated (MA-)SCVP network, trained on a multiview dataset generated by the long-tail sampling method, which addresses the issue of unbalanced multiview inputs and enhances the network performance.

Abstract

Existing view planning systems either adopt an iterative paradigm using next-best views (NBV) or a one-shot pipeline relying on the set-covering view-planning (SCVP) network. However, neither of these methods can concurrently guarantee both high-quality and high-efficiency reconstruction of 3D unknown objects. To tackle this challenge, we introduce a crucial hypothesis: with the availability of more information about the unknown object, the prediction quality of the SCVP network improves. There are two ways to provide extra information: (1) leveraging perception data obtained from NBVs, and (2) training on an expanded dataset of multiview inputs. In this work, we introduce a novel combined pipeline that incorporates a single NBV before activating the proposed multiview-activated (MA-)SCVP network. The MA-SCVP is trained on a multiview dataset generated by our long-tail sampling method, which addresses the issue of unbalanced multiview inputs and enhances the network performance. Extensive simulated experiments substantiate that our system demonstrates a significant surface coverage increase and a substantial 45% reduction in movement cost compared to state-of-the-art systems. Real-world experiments justify the capability of our system for generalization and deployment.

Figures (21)

• Figure 1: Comparative results of reconstructing an unknown object: Each method is depicted through reconstructed 3D models (red point clouds), local paths (cyan), global paths (purple), views (red-green-blue coordinate systems), and the same initial view (black circle). In (a) and (b), it is observed that both the iterative NBV method and the one-shot set-covering view-planning (SCVP) network fail to provide complete surface details (yellow voxels in enlarged areas). To address these missing surfaces, we introduce a novel combined pipeline that incorporates four NBVs before activating the SCVP network, as shown in (c). However, this approach requires more NBVs and paths (4 NBVs with 4 long local paths). Therefore, we propose the multiview-activated MA-SCVP network, trained on our innovative long-tail multiview dataset as shown in (d), requiring only 1 NBV with 1 local path. Our novel pipeline preserves high-quality reconstruction while reducing the number of necessary NBVs and minimizing path length.
• Figure 2: Combined view planning system structure.
• Figure 3: Illustration of the combined view planning pipeline, which performs one NBV planning (NBVP) before activating the MA-SCVP network: The inputs are displayed in OctoMaps to clearly show free areas (green), occupied areas (red object and blue table), and unknown areas (gray voxels near the object). In brief, point clouds and view states are not shown here, which may be required by some view planning methods. The partially reconstructed object models are shown in point cloud space as well as views (red-green-blue), local paths (cyan), and global paths (purple). The output $V^\ast_{cover}$ is the predicted smallest subset covering the remaining object surfaces (views excluding two visited views), which is sequenced to $V_{path}$ by global path planning.
• Figure 4: Illustration of candidate views: (a) The 5-DOF view pose. The last DOF can be regarded as rotating the view pose along the Z+ by any degree (rotating the XvY plane, black rectangle). (b) The view space of 32 candidates.
• Figure 5: Illustration of the view path planning: (a) local path planning (cyan) of a straight path and an obstacle avoidance path; (b) global path planning (purple) by finding the shortest Hamiltonian path on the undirected complete graph of views.

Theorems & Definitions (5)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5