NDRL: Cotton Irrigation and Nitrogen Application with Nested Dual-Agent Reinforcement Learning

TL;DR

This work tackles the complexity of optimizing irrigation and nitrogen for cotton by introducing Nested Dual-Agent Reinforcement Learning (NDRL) integrated with the DSSAT crop model. A two-tier architecture sets macroscopic, two-day macro-actions (parent) and daily refinements (child) using a mixed Gaussian–uniform action distribution, guided by stress signals (WSF/NSF). Results in calibrated DSSAT simulations show consistent yield gains (+4.7%), plus improvements in irrigation water productivity and nitrogen partial factor productivity across 2023–2024, outperforming field data and a DQN baseline in joint optimization. The approach advances precision, resource-efficient crop management and points to scalable opportunities for sustainable agricultural development.

Abstract

Effective irrigation and nitrogen fertilization have a significant impact on crop yield. However, existing research faces two limitations: (1) the high complexity of optimizing water-nitrogen combinations during crop growth and poor yield optimization results; and (2) the difficulty in quantifying mild stress signals and the delayed feedback, which results in less precise dynamic regulation of water and nitrogen and lower resource utilization efficiency. To address these issues, we propose a Nested Dual-Agent Reinforcement Learning (NDRL) method. The parent agent in NDRL identifies promising macroscopic irrigation and fertilization actions based on projected cumulative yield benefits, reducing ineffective explorationwhile maintaining alignment between objectives and yield. The child agent's reward function incorporates quantified Water Stress Factor (WSF) and Nitrogen Stress Factor (NSF), and uses a mixed probability distribution to dynamically optimize daily strategies, thereby enhancing both yield and resource efficiency. We used field experiment data from 2023 and 2024 to calibrate and validate the Decision Support System for Agrotechnology Transfer (DSSAT) to simulate real-world conditions and interact with NDRL. Experimental results demonstrate that, compared to the best baseline, the simulated yield increased by 4.7% in both 2023 and 2024, the irrigation water productivity increased by 5.6% and 5.1% respectively, and the nitrogen partial factor productivity increased by 6.3% and 1.0% respectively. Our method advances the development of cotton irrigation and nitrogen fertilization, providing new ideas for addressing the complexity and precision issues in agricultural resource management and for sustainable agricultural development.

Figures (4)

• Figure 1: Simulated yield and observation results.
• Figure 2: This is the framework of a nested dual-agent reinforcement learning algorithm, which is divided into two parts. The left side represents the optimization process of the child agent, while the right side represents the optimization process of the parent agent. Subscripts beginning with $p$ represent abstractions related to the parent agent, while those beginning with $c$ represent abstractions related to the child agent.
• Figure 3: The distribution of irrigation and nitrogen fertilizer application events over two consecutive years reflects the variability and efficiency of the NDRL algorithm in these practices. The x-axis denotes specific dates of irrigation and nitrogen application, the y-axis represents irrigation depth and nitrogen fertilizer amount, and shaded areas indicate the range corresponding to the child agent.
• Figure 4: The two graphs are the average cumulative reward trends for 2023 and 2024, and the CYASR comparison between these two years. In the average cumulative reward trends, the x-axis shows training episodes and the y-axis shows average cumulative rewards.