Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CF-PRNet: Coarse-to-Fine Prototype Refining Network for Point Cloud Completion and Reconstruction

Zhi Chen, Tianqi Wei, Zecheng Zhao, Jia Syuen Lim, Yadan Luo, Hu Zhang, Xin Yu, Scott Chapman, Zi Huang

TL;DR

CF-PRNet, a coarse-to-fine prototype refining network, leverages high-resolution 3D data during the training phase but requires only a single RGB-D image for real-time inference, and achieves detailed and accurate reconstructions of fruits from partial views.

Abstract

In modern agriculture, precise monitoring of plants and fruits is crucial for tasks such as high-throughput phenotyping and automated harvesting. This paper addresses the challenge of reconstructing accurate 3D shapes of fruits from partial views, which is common in agricultural settings. We introduce CF-PRNet, a coarse-to-fine prototype refining network, leverages high-resolution 3D data during the training phase but requires only a single RGB-D image for real-time inference. Our approach begins by extracting the incomplete point cloud data that constructed from a partial view of a fruit with a series of convolutional blocks. The extracted features inform the generation of scaling vectors that refine two sequentially constructed 3D mesh prototypes - one coarse and one fine-grained. This progressive refinement facilitates the detailed completion of the final point clouds, achieving detailed and accurate reconstructions. CF-PRNet demonstrates excellent performance metrics with a Chamfer Distance of 3.78, an F1 Score of 66.76%, a Precision of 56.56%, and a Recall of 85.31%, and win the first place in the Shape Completion and Reconstruction of Sweet Peppers Challenge.

CF-PRNet: Coarse-to-Fine Prototype Refining Network for Point Cloud Completion and Reconstruction

TL;DR

CF-PRNet, a coarse-to-fine prototype refining network, leverages high-resolution 3D data during the training phase but requires only a single RGB-D image for real-time inference, and achieves detailed and accurate reconstructions of fruits from partial views.

Abstract

In modern agriculture, precise monitoring of plants and fruits is crucial for tasks such as high-throughput phenotyping and automated harvesting. This paper addresses the challenge of reconstructing accurate 3D shapes of fruits from partial views, which is common in agricultural settings. We introduce CF-PRNet, a coarse-to-fine prototype refining network, leverages high-resolution 3D data during the training phase but requires only a single RGB-D image for real-time inference. Our approach begins by extracting the incomplete point cloud data that constructed from a partial view of a fruit with a series of convolutional blocks. The extracted features inform the generation of scaling vectors that refine two sequentially constructed 3D mesh prototypes - one coarse and one fine-grained. This progressive refinement facilitates the detailed completion of the final point clouds, achieving detailed and accurate reconstructions. CF-PRNet demonstrates excellent performance metrics with a Chamfer Distance of 3.78, an F1 Score of 66.76%, a Precision of 56.56%, and a Recall of 85.31%, and win the first place in the Shape Completion and Reconstruction of Sweet Peppers Challenge.
Paper Structure (4 sections, 3 equations, 3 figures, 2 tables)

This paper contains 4 sections, 3 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Experiments
  4. Conclusion

Figures (3)

  • Figure 1: An illustration of the differences between greenhouse and laboratory settings. It can be seen that partial point clouds in greenhouse setting significantly diverge from the ground-truth point clouds. They could not provide similar supervision as the lab setting to the shape completion process, which pose a significant challenge for generalizing the model trained on laboratory data.
  • Figure 2: An illustration of CF-PRNet. The process begins with incomplete point clouds of sweet peppers fed into the feature extractor to obtain global features. These features are then processed by the shape generator to create coarse and dense modifications for the prototypes. The refining module fine-tunes these prototypes, progressively enhancing them from basic to detailed representations.
  • Figure 3: A visualization of the input, coarse and fine-grained output and the GT sample.