Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Reinforcement Learning-Based Telematic Routing Protocol for the Internet of Underwater Things

Mohammadhossein Homaei, Mehran Tarif, Agustin Di Bartolo, Victor Gonzalez Morales, Mar Avila Vegas

TL;DR

RL-RPL-UA tackles the IoUT routing challenge by embedding per-node reinforcement learning agents that adaptively select next hops using a dynamic, RL-driven objective function while remaining compatible with standard RPL messages. The approach models routing as an MDP with state $s_t = [E_t, \text{LQI}_t, Q_t, \text{PDR}_t, T_t]$, actions as neighboring parents, and a reward $r_t = \alpha \cdot \text{PDR}_t - \beta \cdot \text{Delay}_t - \gamma \cdot \text{EnergyCost}_t$, updated via Q-learning. Simulation results in Aqua-Sim show RL-RPL-UA achieving higher PDR, lower end-to-end delay, reduced energy per delivered packet, lower routing overhead, and longer network lifetime than several baselines under static and mobile scenarios. The work provides a practical, scalable routing paradigm for underwater networks, enabling online learning without sacrificing RPL compatibility and with tangible energy-efficiency benefits.

Abstract

The Internet of Underwater Things (IoUT) has a lot of problems, like low bandwidth, high latency, mobility, and not enough energy. Routing protocols that were made for land-based networks, like RPL, don't work well in these underwater settings. This paper talks about RL-RPL-UA, a new routing protocol that uses reinforcement learning to make things work better in underwater situations. Each node has a small RL agent that picks the best parent node depending on local data such the link quality, buffer level, packet delivery ratio, and remaining energy. RL-RPL-UA works with all standard RPL messages and adds a dynamic objective function to help people make decisions in real time. Aqua-Sim simulations demonstrate that RL-RPL-UA boosts packet delivery by up to 9.2%, uses 14.8% less energy per packet, and adds 80 seconds to the network's lifetime compared to previous approaches. These results show that RL-RPL-UA is a potential and energy-efficient way to route data in underwater networks.

A Reinforcement Learning-Based Telematic Routing Protocol for the Internet of Underwater Things

TL;DR

RL-RPL-UA tackles the IoUT routing challenge by embedding per-node reinforcement learning agents that adaptively select next hops using a dynamic, RL-driven objective function while remaining compatible with standard RPL messages. The approach models routing as an MDP with state , actions as neighboring parents, and a reward , updated via Q-learning. Simulation results in Aqua-Sim show RL-RPL-UA achieving higher PDR, lower end-to-end delay, reduced energy per delivered packet, lower routing overhead, and longer network lifetime than several baselines under static and mobile scenarios. The work provides a practical, scalable routing paradigm for underwater networks, enabling online learning without sacrificing RPL compatibility and with tangible energy-efficiency benefits.

Abstract

The Internet of Underwater Things (IoUT) has a lot of problems, like low bandwidth, high latency, mobility, and not enough energy. Routing protocols that were made for land-based networks, like RPL, don't work well in these underwater settings. This paper talks about RL-RPL-UA, a new routing protocol that uses reinforcement learning to make things work better in underwater situations. Each node has a small RL agent that picks the best parent node depending on local data such the link quality, buffer level, packet delivery ratio, and remaining energy. RL-RPL-UA works with all standard RPL messages and adds a dynamic objective function to help people make decisions in real time. Aqua-Sim simulations demonstrate that RL-RPL-UA boosts packet delivery by up to 9.2%, uses 14.8% less energy per packet, and adds 80 seconds to the network's lifetime compared to previous approaches. These results show that RL-RPL-UA is a potential and energy-efficient way to route data in underwater networks.

Paper Structure

This paper contains 32 sections, 17 equations, 10 figures, 3 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. Depth-Based and Clustered Routing
  4. Opportunistic and Game-Theoretic Methods
  5. AI and RL Approaches
  6. Meta-Heuristic and Bio-Inspired Algorithms
  7. RPL-Based Extensions
  8. PROPOSED PROTOCOL: RL-RPL-UA
  9. Protocol Architecture
  10. RL Model
  11. State Space
  12. Action Space
  13. Reward Function
  14. Policy Learning
  15. Adaptive OF
  16. ...and 17 more sections

Figures (10)

  • Figure 1: PDR in the static scenario.
  • Figure 2: PDR in the mobile scenario.
  • Figure 3: End-to-End Delay in the static scenario.
  • Figure 4: End-to-End Delay in the mobile scenario.
  • Figure 5: Energy cost per delivered packet in the static scenario.
  • ...and 5 more figures