Adaptive Sensor Steering Strategy Using Deep Reinforcement Learning for Dynamic Data Acquisition in Digital Twins

TL;DR

This work addresses the challenge of maintaining informative data for digital twins by formulating adaptive sensor steering as a Markov decision process and solving it with a distributional deep reinforcement learning agent. The reward is information-theoretic, based on the Fisher information matrix $Q$ and its determinant, guiding sensor configurations that minimize posterior uncertainty while accounting for spatial error correlations. The framework is implemented in a digital-twin environment (Gymnasium) and validated on a cantilever plate under healthy and damaged conditions, showing improved data acquisition quality and twin accuracy over baselines. Limitations include scalability to more modes/sensors and the assumption of known damage states, with future work focusing on enhanced exploration strategies and online updating integration.

Abstract

This paper introduces a sensor steering methodology based on deep reinforcement learning to enhance the predictive accuracy and decision support capabilities of digital twins by optimising the data acquisition process. Traditional sensor placement techniques are often constrained by one-off optimisation strategies, which limit their applicability for online applications requiring continuous informative data assimilation. The proposed approach addresses this limitation by offering an adaptive framework for sensor placement within the digital twin paradigm. The sensor placement problem is formulated as a Markov decision process, enabling the training and deployment of an agent capable of dynamically repositioning sensors in response to the evolving conditions of the physical structure as represented by the digital twin. This ensures that the digital twin maintains a highly representative and reliable connection to its physical counterpart. The proposed framework is validated through a series of comprehensive case studies involving a cantilever plate structure subjected to diverse conditions, including healthy and damaged conditions. The results demonstrate the capability of the deep reinforcement learning agent to adaptively reposition sensors improving the quality of data acquisition and hence enhancing the overall accuracy of digital twins.

Figures (14)

• Figure 1: The digital twin life cycle which involves continual updating and validation to maintain accuracy in prediction and decision support.
• Figure 2: Evolution of sensor placement strategies for the example of a rectangular plate showing (a) a grid of all possible sensor candidate locations, (b) fixed sensing strategy-- sensors locations are predefined and remain fixed throughout the structure's lifespan, (c) sensor scheduling strategy-- sensors locations are predefined and queried at different times, and (d) sensor steering strategy-- sensors are adapted to new locations of interest across the lifecycle of the structure.
• Figure 3: Graphical model of dynamic decision interaction between physical and digital space showing sensor steering information exchange.
• Figure 4: Sensor configurations score for the 2nd torsional mode shape of a clamped cantilever structure showing (a) four sensors with an efficient spatial distribution given a high score, (b) reduced sensor configuration score due to two close sensors, (c) further reduction in sensor configuration score resulting from two pairs of close sensors, (d) poor spatial distribution of sensor with low configuration score. The reward function takes into consideration the distance between sensors and penalises configuration with low spatial distribution.
• Figure 5: Graphical representation of interaction between physical and digital space. The red cycle indicates the virtual-to-physical and physical-to-virtual interaction; the black cycle shows the interaction between the virtual and reinforcement learning environments and the blue cycle is the agent-environment interaction.