MIS-ME: A Multi-modal Framework for Soil Moisture Estimation

TL;DR

This paper addresses accurate field-scale soil moisture estimation by fusing ground-level soil patch imagery with meteorological data to predict volumetric water content $VWC$. It introduces MIS-ME, a three-way multimodal framework that fuses image features and weather features through (i) Multimodal Concat, (ii) Hybrid Loss, and (iii) Learnable Parameter strategies, delivering superior performance over unimodal baselines. Using a real-world dataset from three Oklahoma stations with YOLOv5-extracted soil patches, MIS-ME achieves a best reported MAPE of $10.14\%$ with Hybrid Loss, improving over meteorological-only and image-only baselines by notable margins (e.g., reductions of $3.25\%$ and $2.15\%$, respectively). The results demonstrate practical potential for real-time, smartphone-like image cues combined with weather data to guide irrigation decisions in precision agriculture.

Abstract

Soil moisture estimation is an important task to enable precision agriculture in creating optimal plans for irrigation, fertilization, and harvest. It is common to utilize statistical and machine learning models to estimate soil moisture from traditional data sources such as weather forecasts, soil properties, and crop properties. However, there is a growing interest in utilizing aerial and geospatial imagery to estimate soil moisture. Although these images capture high-resolution crop details, they are expensive to curate and challenging to interpret. Imagine, an AI-enhanced software tool that predicts soil moisture using visual cues captured by smartphones and statistical data given by weather forecasts. This work is a first step towards that goal of developing a multi-modal approach for soil moisture estimation. In particular, we curate a dataset consisting of real-world images taken from ground stations and their corresponding weather data. We also propose MIS-ME - Meteorological & Image based Soil Moisture Estimator, a multi-modal framework for soil moisture estimation. Our extensive analysis shows that MIS-ME achieves a MAPE of 10.14%, outperforming traditional unimodal approaches with a reduction of 3.25% in MAPE for meteorological data and 2.15% in MAPE for image data, highlighting the effectiveness of tailored multi-modal approaches. Our code and dataset will be available at https://github.com/OSU-Complex-Systems/MIS-ME.git.

Figures (12)

• Figure 1: Integrating cropland images and meteorological data for improved soil moisture estimation and irrigation guidance.
• Figure 2: Overview of the dataset creation pipeline
• Figure 3: Pearson correlation heatmap of the 16 meteorological variables with VWC.
• Figure 4: Trend of VWC from October 2020 to June 2021 of the three stations with Station1 and Station3 showing similar trends compared to Station3.
• Figure 5: MIS-ME with Multimodal Concat extracts soil patch image features using a trained image feature extractor and extracts tabular meteorological features using the proposed MSME model in \ref{['sec:msme']}. The extracted features are combined and batch-normalized to pass through a series of fully connected layers for soil moisture regression.