YieldSAT: A Multimodal Benchmark Dataset for High-Resolution Crop Yield Prediction

Abstract

Crop yield prediction requires substantial data to train scalable models. However, creating yield prediction datasets is constrained by high acquisition costs, heterogeneous data quality, and data privacy regulations. Consequently, existing datasets are scarce, low in quality, or limited to regional levels or single crop types, hindering the development of scalable data-driven solutions. In this work, we release YieldSAT, a large, high-quality, and multimodal dataset for high-resolution crop yield prediction. YieldSAT spans various climate zones across multiple countries, including Argentina, Brazil, Uruguay, and Germany, and includes major crop types, including corn, rapeseed, soybeans, and wheat, across 2,173 expert-curated fields. In total, over 12.2 million yield samples are available, each with a spatial resolution of 10 m. Each field is paired with multispectral satellite imagery, resulting in 113,555 labeled satellite images, complemented by auxiliary environmental data. We demonstrate the potential of large-scale and high-resolution crop yield prediction as a pixel regression task by comparing various deep learning models and data fusion architectures. Furthermore, we highlight open challenges arising from severe distribution shifts in the ground truth data under real-world conditions. To mitigate this, we explore a domain-informed Deep Ensemble approach that exhibits significant performance gains. The dataset is available at https://yieldsat.github.io/.

Figures (19)

• Figure 1: Schematic overview of the YieldSAT dataset and the data collection and preprocessing. At harvest, a combine harvester collects point data containing various information (yield, geolocation, time, and moisture), referred to as a yield map. For each yield map, data is collected (satellite imagery, weather, soil, and topography data). The target yield map and the input data are preprocessed into an ML-ready data format, formulating yield prediction as a pixel-wise regression task.
• Figure 2: Spatial distribution of the collected yield data for each region and crop type. The yield data is colored by crop type. Left: South America, right: Europe. The marker size indicates the field size. (Map source: Cartopy)
• Figure 3: Example satellite time series of a soybean field from seeding (left) to harvesting (right). The last image shows the collected yield at harvest.
• Figure 4: Yield data distribution plots for each crop type and country, averaged per field.
• Figure 5: Examples of rasterized yield maps with the curation for each quality level (good, average, and bad).