AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robot

TL;DR

AgriChrono introduces a field-scale, multi-modal data collection platform and an 18 TB dataset that captures crop growth and lighting variability in real-world farms. The work combines RGB, Depth, LiDAR, IMU, and Pose streams collected over a growth cycle across multiple sites, paired with a seven-scenario 3D reconstruction benchmark to stress-test current methods on non-rigid agricultural scenes. It evaluates NeRF-based and Gaussian Splatting rendering approaches, revealing that state-of-the-art techniques struggle with in-the-wild canopy dynamics and illumination changes, while Gaussian Splatting methods offer favorable speed-accuracy trade-offs in many cases. The dataset and benchmark are publicly released to accelerate robust 3D vision and precision agriculture research, enabling improved digital twins and autonomous field robotics.

Abstract

Advances in AI and Robotics have accelerated significant initiatives in agriculture, particularly in the areas of robot navigation and 3D digital twin creation. A significant bottleneck impeding this progress is the critical lack of "in-the-wild" datasets that capture the full complexities of real farmland, including non-rigid motion from wind, drastic illumination variance, and morphological changes resulting from growth. This data gap fundamentally limits research on robust AI models for autonomous field navigation and scene-level dynamic 3D reconstruction. In this paper, we present AgriChrono, a modular robotic data collection platform and multi-modal dataset designed to capture these dynamic farmland conditions. Our platform integrates multiple sensors, enabling remote, time-synchronized acquisition of RGB, Depth, LiDAR, IMU, and Pose data for efficient and repeatable long-term data collection in real-world agricultural environments. We successfully collected 18TB of data over one month, documenting the entire growth cycle of Canola under diverse illumination conditions. We benchmark state-of-the-art 3D reconstruction methods on AgriChrono, revealing the profound challenge of reconstructing high-fidelity, dynamic non-rigid scenes in such farmland settings. This benchmark validates AgriChrono as a critical asset for advancing model generalization, and its public release is expected to significantly accelerate research and development in precision agriculture. The code and dataset are publicly available at: https://github.com/StructuresComp/agri-chrono

Figures (30)

• Figure 1: Overview of the AgriChrono field-scale data collection framework. Top, a real-world outdoor agricultural field used for data acquisition. Middle left, user interface for remote robot control and data collection. Bottom left, robotic platform with multiple sensors for field data collection. Bottom right, time-aligned multi-modal frame with RGB, Depth, and LiDAR data.
• Figure 2: Data collection sites and the remote operation interface used during field trials. Site 1, main canola site used as the primary location for repeated data collection, capturing temporal changes in illumination and crop structure. Site 2, genotype trial site featuring diverse canola varieties, enabling the capture of morphological variation across crop types. Site 3, flax trial site composed of multiple plots with varying weed control strategies, providing structural diversity in a controlled multi-block layout. WebUI, remote interface supporting real-time system feedback, intuitive directory-based data saving, responsive robot and camera control, and low-latency teleoperation.
• Figure 3: System diagram of the AgriChrono platform, showing the hardware configuration segmented into three modular tiers mounted on a UGV: The first tier houses the DC converter and LiDAR; the second tier contains the Edge Device, 5G Router, and two stereo cameras; and the third tier holds the antennas and the PTZ camera for robot control.
• Figure 4: Visualization of dataset properties. (i) Lighting Variance: RGB frames of the same scene at four times of day, showing natural illumination changes. The top two images are from the left stereo camera, and the bottom two are from the right stereo camera. (ii) Growth Span: Weekly progression (Week 1–3) of the same crop row, highlighting canola growth and structural change. (iii) Time Aligned: Gyroscope magnitude (L2 norm) from IMU data of two ZED cameras and the LiDAR, Z-score normalized to verify cross-sensor alignment.
• Figure 5: Public release format of the AgriChrono dataset