Continual Adversarial Reinforcement Learning (CARL) of False Data Injection detection: forgetting and explainability

TL;DR

It is shown that a continual learning implementation is subject to catastrophic forgetting, and additionally it is shown that forgetting can be addressed by employing a joint training strategy on all generated FDIA scenarios.

Abstract

False data injection attacks (FDIAs) on smart inverters are a growing concern linked to increased renewable energy production. While data-based FDIA detection methods are also actively developed, we show that they remain vulnerable to impactful and stealthy adversarial examples that can be crafted using Reinforcement Learning (RL). We propose to include such adversarial examples in data-based detection training procedure via a continual adversarial RL (CARL) approach. This way, one can pinpoint the deficiencies of data-based detection, thereby offering explainability during their incremental improvement. We show that a continual learning implementation is subject to catastrophic forgetting, and additionally show that forgetting can be addressed by employing a joint training strategy on all generated FDIA scenarios.

Figures (4)

• Figure 1: (i)Schematic of the CARL process, (ii) framework of CARL process.
• Figure 2: Left: bus under FDIA from $A_1$ (pythonBlue) and corresponding successful detection from $D_0$ (blackwhite) and $D_1$ (blackred). Right: time-series of frequency deviations induced for each bus induced by $A_1$.
• Figure 3: Transition matrix differences (Eq. \ref{['eq:dist']}) between pairs of adversaries.
• Figure 4: Histogram of droop coefficient modification for all adversaries.