CropCraft: Inverse Procedural Modeling for 3D Reconstruction of Crop Plants

TL;DR

CropCraft introduces an inverse procedural modeling framework to reconstruct complete 3D crop canopies from images by fitting a low-dimensional, biologically informed morphology model. The method combines neural radiance field-based depth estimation with row-aligned depth rendering and Bayesian optimization to recover occluded canopy structure while maintaining physical plausibility. Validation on real soybean and maize field data demonstrates accurate canopy-level traits (LAI, leaf angle) and enables direct integration with radiative transfer models for photosynthesis simulations, highlighting practical utility for crop monitoring and productivity analysis. This work bridges data-driven surface reconstruction with physics-informed generative models, offering a scalable approach to field-ready crop 3D reconstruction and simulation.

Abstract

The ability to automatically build 3D digital twins of plants from images has countless applications in agriculture, environmental science, robotics, and other fields. However, current 3D reconstruction methods fail to recover complete shapes of plants due to heavy occlusion and complex geometries. In this work, we present a novel method for 3D reconstruction of agricultural crops based on optimizing a parametric model of plant morphology via inverse procedural modeling. Our method first estimates depth maps by fitting a neural radiance field and then employs Bayesian optimization to estimate plant morphological parameters that result in consistent depth renderings. The resulting 3D model is complete and biologically plausible. We validate our method on a dataset of real images of agricultural fields, and demonstrate that the reconstructions can be used for a variety of monitoring and simulation applications.

Figures (9)

• Figure 1: Inverse procedural modeling for agricultural crops. We propose a novel method for 3D reconstruction of agricultural crops based on inverse procedural modeling. Unlike standard multi-view reconstruction pipelines, our method outputs a complete, interpretable, and biologically plausible 3D mesh model of the crop canopy, lending itself to simulations of important biophysical processes such as photosynthesis.
• Figure 2: Overview of our method. We aim to estimate the parameters for a procedural generation model to generate a shape that matches the observed images. First, we use standard structure-from-motion and neural radiance field (NeRF) techniques to reconstruct the visible geometry of the scene. We then apply RANSAC to acquire a camera pose aligned with the planting rows of the crops. This pose is used to render depth maps from both the NeRF and the procedural model. We define a loss function based on histogram statistics of the depth maps and minimize it with respect to the morphological parameters using Bayesian optimization.
• Figure 3: Procedural plant morphology models. We adopt procedural generation models from existing soybean morphology song2020decomposition and maize morphology qian2023coupled models. This (stylized) illustration shows the parameters that we allow to be optimized to model variations across individual instances of each species.
• Figure 4: Qualitative reconstruction results (soybean). We validate the reconstruction quality from our method on images from real agricultural fields. From left to right: example images from the multi-view data, row-aligned NeRF-rendered depth, procedural model depth, rendered visualization of the procedural mesh. By matching statistics of the predicted depth with the observed depth, our method is able to estimate the key shape parameters needed to characterize the growth of the plants throughout the growing season, consistently producing realistic reconstructions.
• Figure 5: Qualitative reconstruction results (maize). We show that our method can be applied to model maize as well as soybean. From left to right: example images from the drone-captured multi-view data, row-aligned NeRF-rendered depth, procedural model depth, rendered visualization of the procedural mesh. The resulting reconstructions are complete and anatomically realistic despite heavy occlusion.