Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A semi-centralized multi-agent RL framework for efficient irrigation scheduling

Bernard T. Agyeman, Benjamin Decard-Nelson, Jinfeng Liu, Sirish L. Shah

TL;DR

SCMARL proposes a two-tier semi-centralized MARL framework for irrigation scheduling in spatially variable fields divided into Management Zones, with a centralized coordinator issuing daily binary irrigation decisions and decentralized local agents setting zone-specific amounts. The approach uses state augmentation to address MARL non-stationarity and PPO for learning, achieving improved field-wide coordination over fully decentralized baselines. In a large-scale field, SCMARL reduces total irrigation by 4.0% and increases Irrigation Water Use Efficiency (IWUE) by 6.3% relative to a learning-based multi-agent MPC baseline, while enabling substantial water savings and robust learning. The work also demonstrates that communicating the coordinator’s decision to local agents enhances learning stability, and it discusses potential integrations with MPC to further enhance performance and scalability.

Abstract

This paper proposes a Semi-Centralized Multi-Agent Reinforcement Learning (SCMARL) approach for irrigation scheduling in spatially variable agricultural fields, where management zones address spatial variability. The SCMARL framework is hierarchical in nature, with a centralized coordinator agent at the top level and decentralized local agents at the second level. The coordinator agent makes daily binary irrigation decisions based on field-wide conditions, which are communicated to the local agents. Local agents determine appropriate irrigation amounts for specific management zones using local conditions. The framework employs state augmentation approach to handle non-stationarity in the local agents' environments. An extensive evaluation on a large-scale field in Lethbridge, Canada, compares the SCMARL approach with a learning-based multi-agent model predictive control scheduling approach, highlighting its enhanced performance, resulting in water conservation and improved Irrigation Water Use Efficiency (IWUE). Notably, the proposed approach achieved a 4.0% savings in irrigation water while enhancing the IWUE by 6.3%.

A semi-centralized multi-agent RL framework for efficient irrigation scheduling

TL;DR

SCMARL proposes a two-tier semi-centralized MARL framework for irrigation scheduling in spatially variable fields divided into Management Zones, with a centralized coordinator issuing daily binary irrigation decisions and decentralized local agents setting zone-specific amounts. The approach uses state augmentation to address MARL non-stationarity and PPO for learning, achieving improved field-wide coordination over fully decentralized baselines. In a large-scale field, SCMARL reduces total irrigation by 4.0% and increases Irrigation Water Use Efficiency (IWUE) by 6.3% relative to a learning-based multi-agent MPC baseline, while enabling substantial water savings and robust learning. The work also demonstrates that communicating the coordinator’s decision to local agents enhances learning stability, and it discusses potential integrations with MPC to further enhance performance and scalability.

Abstract

This paper proposes a Semi-Centralized Multi-Agent Reinforcement Learning (SCMARL) approach for irrigation scheduling in spatially variable agricultural fields, where management zones address spatial variability. The SCMARL framework is hierarchical in nature, with a centralized coordinator agent at the top level and decentralized local agents at the second level. The coordinator agent makes daily binary irrigation decisions based on field-wide conditions, which are communicated to the local agents. Local agents determine appropriate irrigation amounts for specific management zones using local conditions. The framework employs state augmentation approach to handle non-stationarity in the local agents' environments. An extensive evaluation on a large-scale field in Lethbridge, Canada, compares the SCMARL approach with a learning-based multi-agent model predictive control scheduling approach, highlighting its enhanced performance, resulting in water conservation and improved Irrigation Water Use Efficiency (IWUE). Notably, the proposed approach achieved a 4.0% savings in irrigation water while enhancing the IWUE by 6.3%.
Paper Structure (40 sections, 26 equations, 10 figures, 4 tables, 1 algorithm)

This paper contains 40 sections, 26 equations, 10 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Formulation
  4. System Description
  5. Proposed Approach: Semi-Centralized MARL
  6. Non-stationarity in the SCMARL Framework
  7. Local Agent Design
  8. Transition Dynamics
  9. State Design
  10. Reward Design
  11. Coordinator Agent Design
  12. Transition Dynamics
  13. State Design
  14. Reward Design
  15. Experimental Setup and Design
  16. ...and 25 more sections

Figures (10)

  • Figure 1: A diagrammatic representation of a field with variability in crop and soil types. The field is divided into 4 distinct MZs, each with uniform soil and crop properties.
  • Figure 2: A schematic diagram of the Semi-Centralized MARL framework for irrigation scheduling.
  • Figure 3: Study area and its management zone map.
  • Figure 4: A schematic representation of the daily evaluation of the learning-based multi-agent MPC approach during the season-long investigation.
  • Figure 5: A schematic representation of the daily evaluation of the SCMARL framework during the season-long investigation.
  • ...and 5 more figures