MARLIN: Multi-Agent Reinforcement Learning with Murmuration Intelligence and LLM Guidance for Reservoir Management

TL;DR

MARLIN tackles the dual challenge of cascading physical transfer uncertainty and environmental/human uncertainty in large reservoir networks by marrying murmuration-inspired decentralized MARL with LLM-guided reward shaping. The approach enables emergent, robust coordination among reservoirs while adaptively incorporating textual information from forecasts, regulations, and stakeholder inputs. Empirical results on real USGS data show improved uncertainty handling, reduced computation, and faster flood response, with demonstrated scalability to networks of up to 10,000 nodes and 1.85 s decision times. The work highlights emergence, adaptive coordination, and uncertainty-driven robustness as key benefits for disaster prevention and water security in complex hydrologic systems.

Abstract

As climate change intensifies extreme weather events, water disasters pose growing threats to global communities, making adaptive reservoir management critical for protecting vulnerable populations and ensuring water security. Modern water resource management faces unprecedented challenges from cascading uncertainties propagating through interconnected reservoir networks. These uncertainties, rooted in physical water transfer losses and environmental variability, make precise control difficult. For example, sending 10 tons downstream may yield only 8-12 tons due to evaporation and seepage. Traditional centralized optimization approaches suffer from exponential computational complexity and cannot effectively handle such real-world uncertainties, while existing multi-agent reinforcement learning (MARL) methods fail to achieve effective coordination under uncertainty. To address these challenges, we present MARLIN, a decentralized reservoir management framework inspired by starling murmurations intelligence. Integrating bio-inspired alignment, separation, and cohesion rules with MARL, MARLIN enables individual reservoirs to make local decisions while achieving emergent global coordination. In addition, a LLM provides real-time reward shaping signals, guiding agents to adapt to environmental changes and human-defined preferences. Experiments on real-world USGS data show that MARLIN improves uncertainty handling by 23\%, cuts computation by 35\%, and accelerates flood response by 68\%, exhibiting super-linear coordination, with complexity scaling 5.4x from 400 to 10,000 nodes. These results demonstrate MARLIN's potential for disaster prevention and protecting communities through intelligent, scalable water resource management.

Figures (11)

• Figure 1: Water resource management challenges: (a) complex networks of interconnected reservoirs requiring coordination, (b) individual reservoirs must be locally managed, and (c) high global flood frequency demanding robust management systems.
• Figure 2: Illustration of centralized vs. decentralized coordination under uncertainty. The target is for node C to receive 10 tons of water. In the centralized baseline, node A releases 25 tons based on static planning, yet due to environmental variations like rainfall, node C ultimately receives only 6 tons. In contrast, MARLIN decentralized coordination allows each node to adapt through local feedback: node B first releases 12 tons, and node A subsequently compensates by adjusting its output to 35 tons, guided by LLM-assisted local adjustments. This closed-loop adaptation enables node C to obtain 10 tons despite environmental uncertainty.
• Figure 3: Starling Murmuration: emergent intelligence from simple local coordination rules.
• Figure 4: Statewide reservoir status under uncertainty: Initial vs. Centralized Baseline vs. Decentralized MARLIN. Each panel shows reservoir water levels. The centralized baseline yields only modest stabilization, whereas MARLIN achieves near-optimal balance with lower variance and better regional coordination.
• Figure 5: Illustration of the multi-layer reservoir network system under cascading uncertainty. Upstream environmental disturbances (e.g., heat, rainfall, freezing, human activities) dynamically affect inflows, leading to transfer uncertainty among interconnected reservoirs. Each node (e.g., node A) operates under both environmental and flow-level uncertainties. The right panel shows the murmuration-inspired coordination rules (alignment, separation, and cohesion) that address the Level 1: Physical Transfer Uncertainty. The LLM-assisted reward shaping layer translates human, industrial, and ecological objectives into adaptive decision feedback, addressing Level 2: Environmental and Human Modulation.

Theorems & Definitions (2)

• Remark 1
• Remark 2