Deep Reinforcement Learning for Digital Twin-Oriented Complex Networked Systems

TL;DR

These findings indicate that promoting cooperation and reducing free-riders can improve public health during epidemics, and the temporal DT-CNS model, where reinforcement learning-driven nodes make decisions on temporal directed interactions in an epidemic outbreak is proposed.

Abstract

The Digital Twin Oriented Complex Networked System (DT-CNS) aims to build and extend a Complex Networked System (CNS) model with progressively increasing dynamics complexity towards an accurate reflection of reality -- a Digital Twin of reality. Our previous work proposed evolutionary DT-CNSs to model the long-term adaptive network changes in an epidemic outbreak. This study extends this framework by proposeing the temporal DT-CNS model, where reinforcement learning-driven nodes make decisions on temporal directed interactions in an epidemic outbreak. We consider cooperative nodes, as well as egocentric and ignorant "free-riders" in the cooperation. We describe this epidemic spreading process with the Susceptible-Infected-Recovered ($SIR$) model and investigate the impact of epidemic severity on the epidemic resilience for different types of nodes. Our experimental results show that (i) the full cooperation leads to a higher reward and lower infection number than a cooperation with egocentric or ignorant "free-riders"; (ii) an increasing number of "free-riders" in a cooperation leads to a smaller reward, while an increasing number of egocentric "free-riders" further escalate the infection numbers and (iii) higher infection rates and a slower recovery weakens networks' resilience to severe epidemic outbreaks. These findings also indicate that promoting cooperation and reducing "free-riders" can improve public health during epidemics.

Figures (7)

• Figure 1: Generations of DT-CNSs, including generation 1: dynamic process on static networks, generation 2: evolving dynamic process on evolving networks, generation 3: evolving dynamic processes on evolving networks with interrelations between them, generation 4: temporal dynamic processes on temporal networks with interrelations between them and the acquisition of real time information, and generation 5 (a DT): Temporal dynamic processes on temporal networks with interrelations between them, as well as the real time two-way feedback between the reality and the CNSs, enabling an idealised state required by a DT.
• Figure 2: The cumulative reward in cooperative, ignorant and egocentric scenarios given various setups of infection rate and recovery time.
• Figure 3: The cumulative reward in the network simulations driven by the mixed preference mutation styles
• Figure 4: The cumulative infection occurrences in cooperative, ignorant and egocentric scenarios given various setups of infection rate and recovery time.
• Figure 5: The cumulative total reward in cooperative, ignorant and egocentric scenarios given various setups of infection rate and recovery time.