MMCBE: Multi-modality Dataset for Crop Biomass Prediction and Beyond

TL;DR

MMCBE introduces a publicly available, multi-modality dataset for high-resolution crop biomass estimation by combining 216 multi-view drone image sets with LiDAR point clouds and hand-labelled ground truth. The paper establishes benchmarks for biomass prediction and evaluates state-of-the-art methods, while also exploring auxiliary tasks such as 3D reconstruction and novel-view rendering. Key contributions include the first open multi-modality crop biomass dataset, a detailed data collection and processing pipeline, and an assessment of both biomass estimation and related vision tasks. This dataset addresses data scarcity in agro-vision, enabling more robust, scalable, and interoperable biomass estimation and informing future UAV-LiDAR fusion research in agriculture.

Abstract

Crop biomass, a critical indicator of plant growth, health, and productivity, is invaluable for crop breeding programs and agronomic research. However, the accurate and scalable quantification of crop biomass remains inaccessible due to limitations in existing measurement methods. One of the obstacles impeding the advancement of current crop biomass prediction methodologies is the scarcity of publicly available datasets. Addressing this gap, we introduce a new dataset in this domain, i.e. Multi-modality dataset for crop biomass estimation (MMCBE). Comprising 216 sets of multi-view drone images, coupled with LiDAR point clouds, and hand-labelled ground truth, MMCBE represents the first multi-modality one in the field. This dataset aims to establish benchmark methods for crop biomass quantification and foster the development of vision-based approaches. We have rigorously evaluated state-of-the-art crop biomass estimation methods using MMCBE and ventured into additional potential applications, such as 3D crop reconstruction from drone imagery and novel-view rendering. With this publication, we are making our comprehensive dataset available to the broader community.

Figures (10)

• Figure 1: The experimental field for data collection.
• Figure 2: The visualization of how experimental fields change over time. The early to late growth stages of crops over the life cycle are presented from left to right.
• Figure 3: How manually measured above-ground biomass of each plot changes over time. For each time point, there are 24 sampling points, with four repetitions and six varieties. Each variety is marked with the same colour.
• Figure 4: The histogram of the manually measured above-ground biomass.
• Figure 5: The processing pipeline of drone images: all drone images are input into SfM algorithms schonberger2016structure for 3D reconstruction and camera poses estimation, then we manually selected 28 points, marked with yellow dots, to locate each crop sample, and we can get drone multi-view images for each sample by calculating the distance between manually-selected points and camera poses. We finally crop 24 3D reconstructed point clouds and collect 24 multi-view image sets.