iTRPL: An Intelligent and Trusted RPL Protocol based on Multi-Agent Reinforcement Learning
Debasmita Dey, Nirnay Ghosh
TL;DR
iTRPL presents a trust-based, multi-agent reinforcement learning framework to mitigate insider threats in RPL-created DODAGs. By computing direct and indirect trust and feeding these signals into a cooperative ε-Greedy MARL scheme at the root (with per-episode and epoch-level policies), the approach dynamically decides whether to retain or modify the DODAG, thereby preventing compromised nodes from degrading network performance. The framework introduces an Inverse Gompertz-based trust model, hierarchical trust provisioning, and a detailed MARL flow that handles non-root and root decisions, validated through a Python-based simulation across varying malicious-node scenarios. Results show that iTRPL can learn to optimize retention vs modification decisions over time, balancing security and network continuity, with potential applicability to scalable, decentralized IoT deployments. The work highlights the significance of integrating soft security with RPL’s authentication to address insider threats in resource-constrained networks and points to future work involving multiple connected DODAGs and multiple roots.
Abstract
Routing Protocol for Low Power and Lossy Networks (RPL) is the de-facto routing standard in IoT networks. It enables nodes to collaborate and autonomously build ad-hoc networks modeled by tree-like destination-oriented direct acyclic graphs (DODAG). Despite its widespread usage in industry and healthcare domains, RPL is susceptible to insider attacks. Although the state-of-the-art RPL ensures that only authenticated nodes participate in DODAG, such hard security measures are still inadequate to prevent insider threats. This entails a need to integrate soft security mechanisms to support decision-making. This paper proposes iTRPL, an intelligent and behavior-based framework that incorporates trust to segregate honest and malicious nodes within a DODAG. It also leverages multi-agent reinforcement learning (MARL) to make autonomous decisions concerning the DODAG. The framework enables a parent node to compute the trust for its child and decide if the latter can join the DODAG. It tracks the behavior of the child node, updates the trust, computes the rewards (or penalties), and shares with the root. The root aggregates the rewards/penalties of all nodes, computes the overall return, and decides via its $ε$-Greedy MARL module if the DODAG will be retained or modified for the future. A simulation-based performance evaluation demonstrates that iTRPL learns to make optimal decisions with time.