AI in Energy Digital Twining: A Reinforcement Learning-based Adaptive Digital Twin Model for Green Cities

TL;DR

The paper tackles accurate, energy-efficient Digital Twins for green cities under dynamic transportation demands by combining spatiotemporal graphs with a Reinforcement Learning-based Adaptive Twining (RL-AT) policy that uses a Deep Q Network (DQN). Data are modeled as a graph signal $X_t\in\mathbb{R}^{N\times F\times T}$ on a graph $G=(V,E,w)$ and stored in a graph database (Neo4j), while updates are selectively triggered by the RL agent to optimize the trade-off between accuracy and memory-energy cost. The reward balances the change between consecutive snapshots against a memory-operation penalty, guiding the agent to update the most impactful nodes. Empirical results show a 55% speedup in graph querying, 20% RAM savings, and 25% energy reductions, demonstrating improved DT accuracy and synchronization suitable for scalable green-city deployments.

Abstract

Digital Twins (DT) have become crucial to achieve sustainable and effective smart urban solutions. However, current DT modelling techniques cannot support the dynamicity of these smart city environments. This is caused by the lack of right-time data capturing in traditional approaches, resulting in inaccurate modelling and high resource and energy consumption challenges. To fill this gap, we explore spatiotemporal graphs and propose the Reinforcement Learning-based Adaptive Twining (RL-AT) mechanism with Deep Q Networks (DQN). By doing so, our study contributes to advancing Green Cities and showcases tangible benefits in accuracy, synchronisation, resource optimization, and energy efficiency. As a result, we note the spatiotemporal graphs are able to offer a consistent accuracy and 55% higher querying performance when implemented using graph databases. In addition, our model demonstrates right-time data capturing with 20% lower overhead and 25% lower energy consumption.

Figures (6)

• Figure 1: Reinforcement Learning-based Adaptive Digital Twin Model for Green Cities
• Figure 2: RL-AT Mechanism
• Figure 3: Querying latency for proposed spatio-temporal graphs and relational databases.
• Figure 4: Cumulative reward function for DQN training episodes
• Figure 5: Average energy consumption (mJ) and RAM usage percentage comparison under changing payload sizes with and without RL-based Adaptive Twining

Theorems & Definitions (2)

• Definition 3.1: Spatial Graph, $S$
• Definition 3.2: Graph Signal, $X$