Adversarial Multi-Agent Reinforcement Learning for Proactive False Data Injection Detection

TL;DR

This work addresses FDIA on inverter-based grid assets by formulating a proactive defense as a competitive MARL problem between a defender and an adversary. The defender learns to locate attacked inverters, while the adversary generates unseen FDIAs to disrupt operation, with a detection window guiding credit assignment. A transfer-learning approach initializes the MARL defender from an offline, supervised defender to improve robustness against unseen attacks, and is demonstrated on voltage-regulation and frequency-control case studies. Results show that the MARL defender outperforms the offline baseline and that transfer learning enables reliable detection of synthetic and adversarial FDIAs, highlighting practical benefits for cyber-physical grid security. The method offers a scalable, data-driven defense against evolving FDIA threats in both distribution and transmission networks, though careful management of warm-start to avoid forgetting remains important.

Abstract

Smart inverters are instrumental in the integration of distributed energy resources into the electric grid. Such inverters rely on communication layers for continuous control and monitoring, potentially exposing them to cyber-physical attacks such as false data injection attacks (FDIAs). We propose to construct a defense strategy against a priori unknown FDIAs with a multi-agent reinforcement learning (MARL) framework. The first agent is an adversary that simulates and discovers various FDIA strategies, while the second agent is a defender in charge of detecting and locating FDIAs. This approach enables the defender to be trained against new FDIAs continuously generated by the adversary. In addition, we show that the detection skills of an MARL defender can be combined with those of a supervised offline defender through a transfer learning approach. Numerical experiments conducted on a distribution and transmission system demonstrate that: a) the proposed MARL defender outperforms the offline defender against adversarial attacks; b) the transfer learning approach makes the MARL defender capable against both synthetic and unseen FDIAs.

Figures (14)

• Figure 1: Multi-agent RL framework for the FDIA detection.
• Figure 2: Schematic illustration of the detection scheme.
• Figure 3: Illustration scheme of the VW and VV control functions.
• Figure 4: The structure of the inverter and the FDIA scheme.
• Figure 5: Multi-agent framework for the FDIA detection of voltage control in the distribution grid.