Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PointRAFT: 3D deep learning for high-throughput prediction of potato tuber weight from partial point clouds

Pieter M. Blok, Haozhou Wang, Hyun Kwon Suh, Peicheng Wang, James Burridge, Wei Guo

TL;DR

PointRAFT addresses the challenge of estimating potato tuber weight from partially observed RGB-D point clouds, caused by self-occlusion on conveyors. It introduces a high-throughput regression network based on a PointNet++ backbone with an object height embedding that encodes approximate tuber height, yielding accurate weight predictions directly from partial data. On a large, multi-season field dataset, PointRAFT achieves an overall MAE of $12.0$ g and RMSE of $17.2$ g, outperforming a linear regression baseline and previous 3D completion methods, with inference times around $6.3$ ms per cloud and throughput up to $150$ tubers/s. The approach demonstrates strong generalization across cultivars and camera placements and offers a versatile encoder for 3D phenotyping tasks, while outlining future work to handle stacking, orientation, and uncertainty estimation for deployment in commercial harvesters.

Abstract

Potato yield is a key indicator for optimizing cultivation practices in agriculture. Potato yield can be estimated on harvesters using RGB-D cameras, which capture three-dimensional (3D) information of individual tubers moving along the conveyor belt. However, point clouds reconstructed from RGB-D images are incomplete due to self-occlusion, leading to systematic underestimation of tuber weight. To address this, we introduce PointRAFT, a high-throughput point cloud regression network that directly predicts continuous 3D shape properties, such as tuber weight, from partial point clouds. Rather than reconstructing full 3D geometry, PointRAFT infers target values directly from raw 3D data. Its key architectural novelty is an object height embedding that incorporates tuber height as an additional geometric cue, improving weight prediction under practical harvesting conditions. PointRAFT was trained and evaluated on 26,688 partial point clouds collected from 859 potato tubers across four cultivars and three growing seasons on an operational harvester in Japan. On a test set of 5,254 point clouds from 172 tubers, PointRAFT achieved a mean absolute error of 12.0 g and a root mean squared error of 17.2 g, substantially outperforming a linear regression baseline and a standard PointNet++ regression network. With an average inference time of 6.3 ms per point cloud, PointRAFT supports processing rates of up to 150 tubers per second, meeting the high-throughput requirements of commercial potato harvesters. Beyond potato weight estimation, PointRAFT provides a versatile regression network applicable to a wide range of 3D phenotyping and robotic perception tasks. The code, network weights, and a subset of the dataset are publicly available at https://github.com/pieterblok/pointraft.git.

PointRAFT: 3D deep learning for high-throughput prediction of potato tuber weight from partial point clouds

TL;DR

PointRAFT addresses the challenge of estimating potato tuber weight from partially observed RGB-D point clouds, caused by self-occlusion on conveyors. It introduces a high-throughput regression network based on a PointNet++ backbone with an object height embedding that encodes approximate tuber height, yielding accurate weight predictions directly from partial data. On a large, multi-season field dataset, PointRAFT achieves an overall MAE of g and RMSE of g, outperforming a linear regression baseline and previous 3D completion methods, with inference times around ms per cloud and throughput up to tubers/s. The approach demonstrates strong generalization across cultivars and camera placements and offers a versatile encoder for 3D phenotyping tasks, while outlining future work to handle stacking, orientation, and uncertainty estimation for deployment in commercial harvesters.

Abstract

Potato yield is a key indicator for optimizing cultivation practices in agriculture. Potato yield can be estimated on harvesters using RGB-D cameras, which capture three-dimensional (3D) information of individual tubers moving along the conveyor belt. However, point clouds reconstructed from RGB-D images are incomplete due to self-occlusion, leading to systematic underestimation of tuber weight. To address this, we introduce PointRAFT, a high-throughput point cloud regression network that directly predicts continuous 3D shape properties, such as tuber weight, from partial point clouds. Rather than reconstructing full 3D geometry, PointRAFT infers target values directly from raw 3D data. Its key architectural novelty is an object height embedding that incorporates tuber height as an additional geometric cue, improving weight prediction under practical harvesting conditions. PointRAFT was trained and evaluated on 26,688 partial point clouds collected from 859 potato tubers across four cultivars and three growing seasons on an operational harvester in Japan. On a test set of 5,254 point clouds from 172 tubers, PointRAFT achieved a mean absolute error of 12.0 g and a root mean squared error of 17.2 g, substantially outperforming a linear regression baseline and a standard PointNet++ regression network. With an average inference time of 6.3 ms per point cloud, PointRAFT supports processing rates of up to 150 tubers per second, meeting the high-throughput requirements of commercial potato harvesters. Beyond potato weight estimation, PointRAFT provides a versatile regression network applicable to a wide range of 3D phenotyping and robotic perception tasks. The code, network weights, and a subset of the dataset are publicly available at https://github.com/pieterblok/pointraft.git.
Paper Structure (24 sections, 4 equations, 7 figures, 3 tables)

This paper contains 24 sections, 4 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Materials and methods
  3. PointRAFT
  4. Preprocessing of partial point clouds
  5. Neural network architecture
  6. Dataset
  7. Imaging systems
  8. Data collection on the harvester
  9. Point cloud creation
  10. Dataset splits
  11. Neural network training
  12. Embedding of tuber height
  13. Data augmentation
  14. Data sampler for imbalanced dataset
  15. Training details
  16. ...and 9 more sections

Figures (7)

  • Figure 1: Schematic representation of the PointRAFT network, which directly regresses continuous 3D shape properties, such as tuber weight, from partial point clouds. PointRAFT’s architectural novelty lies in an object height embedding, which provides an additional geometric cue for weight estimation.
  • Figure 2: (a) and (b) provide overviews of the first imaging system, which was installed on a single-row potato harvester and used during the 2023 growing season. (c) Inside the imaging box, an RGB-D camera was installed, together with four LED strips that provided the necessary illumination inside the box. The sides of box were covered with a reflective curtain to diffuse the light evenly across the conveyor belt.
  • Figure 3: (a) Overview of the second imaging system, which was used during the 2024 and 2025 growing seasons. One side plate has been removed to reveal the internal components. (b) The yield monitoring system with enclosed sides during operation on the harvester. (c) On the ceiling plate of the enclosure, 24 LED strips and the RGB-D camera were installed.
  • Figure 4: Kernel density estimate plot for visualizing the weight distribution in the training, validation, and test set.
  • Figure 5: Predicted weights of both methods on the test samples. The blue dots are the predictions from PointRAFT and the red crosses are the predictions from the linear regression benchmark.
  • ...and 2 more figures