Fréchet Power-Scenario Distance: A Metric for Evaluating Generative AI Models across Multiple Time-Scales in Smart Grids

TL;DR

This work tackles the problem of evaluating generative AI outputs for smart grids across multiple time scales, where traditional sample-wise metrics fail to capture temporal and physical structure. It introduces the Fréchet Power-Scenario Distance (FPD), a distributional metric computed in a power-aware, hierarchical, multi-resolution feature space that encodes grid-specific temporal patterns and constraints. The approach comprises a novel power-aware feature extractor and a Fréchet-distance-based comparison of real versus generated feature distributions, validated across disturbances, resolutions, and downstream tasks. The results demonstrate that FPD provides robust, interpretable, and task-aligned assessments, supporting standardized benchmarking and more reliable data-driven decision-making in smart grid operations.

Abstract

Generative artificial intelligence (AI) models in smart grids have advanced significantly in recent years due to their ability to generate large amounts of synthetic data, which would otherwise be difficult to obtain in the real world due to confidentiality constraints. A key challenge in utilizing such synthetic data is how to assess the data quality produced from such generative models. Traditional Euclidean distance-based metrics only reflect pair-wise relations between two individual samples, and could fail in evaluating quality differences between groups of synthetic datasets. In this work, we propose a novel metric based on the Fréchet Distance (FD) estimated between two datasets in a learned feature space. The proposed method evaluates the quality of generation from a distributional perspective. Empirical results demonstrate the superiority of the proposed metric across timescales and models, enhancing the reliability of data-driven decision-making in smart grid operations.

Figures (5)

• Figure 1: Overview of the hierarchical multi-resolution feature extraction framework, integrating multiple time-scale feature extractors to enhance synthetic data evaluation in smart grids.
• Figure 2: Features from the proposed power-aware extractor vs. generic time-series encoders, evaluated using the Fréchet distance in feature space in case: (a) Period Offset: solar series are shifted by $\alpha \in \{0,2,4\}$ hours. (b) Solar Physical Violation: Unrealistic power generation is injected for $\alpha \in \{0,2,3\}$ hours of solar generation during night. (c) Cross Time Resolution: 5-minute solar compared against hourly solar/wind.
• Figure 3: Effectiveness of FPD with six different types of disturbances on the dataset $X$: (a) Gaussian Noise is added with mean 0 and varying variances $\alpha = [0, 0.16, 1.6, 4]$, where a larger $\alpha$ indicates more noise. (b) Missing Data is simulated by randomly replacing $\alpha \times 100\%$ of the points in each sample of $X$ with zeros, where $\alpha = [0, 0.1, 0.25, 0.5]$. (c) Solar Data Contamination replaces $\alpha \times 100\%$ of the points in $X$ with corresponding points from a solar power dataset $Y$nrel_sind_toolkit, where $\alpha = [0, 0.25, 0.5, 0.75]$. (d) Gaussian Smooth applies Gaussian filter which has variance $\alpha = [0, 10, 20, 30]$ to the original dataset. A larger $\alpha$ indicates more significant smoothing. (e) Error Cumulate is simulated by adding an error, which is multipled by a random Gaussian noise with a mean of 1 and varying variances $\alpha = [0, 0.005, 0.01, 0.03]$ at each time step. (f) Time Shift is a potential problem that exists in time-series generation. To simulate this effect, we shift the data forward by $\alpha = [0,40,60,80]$ intervals.
• Figure 4: Sample-wise distance MAPE vs. FPD
• Figure 5: Data-level distribution metrics vs. FPD