Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models

Kaustav Bhattacharjee, Soumya Kundu, Indrasis Chakraborty, Aritra Dasgupta

TL;DR

This work tackles the challenge of trusting net-load forecasting by enabling post-hoc, visual-analytic comparison between a deep-learning probabilistic forecast and a simple reference benchmark across varying solar penetration, data resolutions, and times of day. It presents a kernelized probabilistic model with an autoencoder and LSTM, later simplified to an LSTM-only variant to better handle lower-resolution data, and evaluates performance using $CRPS$ and $CRPSS$. The core contribution is a coordinated, interactive interface with a Comparison View and a Patterns View that helps researchers and grid operators interpret model behavior under diverse conditions and timeframes, thereby enhancing trust in forecasts. Case study insights demonstrate the tool’s ability to reveal when high-resolution data improves performance, identify diurnal-pattern dependencies, and guide deployment decisions, with plans to extend the platform to multiple models and economics-focused analyses for real-world applicability.

Abstract

Net load forecasting is crucial for energy planning and facilitating informed decision-making regarding trade and load distributions. However, evaluating forecasting models' performance against benchmark models remains challenging, thereby impeding experts' trust in the model's performance. In this context, there is a demand for technological interventions that allow scientists to compare models across various timeframes and solar penetration levels. This paper introduces a visual analytics-based application designed to compare the performance of deep-learning-based net load forecasting models with other models for probabilistic net load forecasting. This application employs carefully selected visual analytic interventions, enabling users to discern differences in model performance across different solar penetration levels, dataset resolutions, and hours of the day over multiple months. We also present observations made using our application through a case study, demonstrating the effectiveness of visualizations in aiding scientists in making informed decisions and enhancing trust in net load forecasting models.

Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models

TL;DR

This work tackles the challenge of trusting net-load forecasting by enabling post-hoc, visual-analytic comparison between a deep-learning probabilistic forecast and a simple reference benchmark across varying solar penetration, data resolutions, and times of day. It presents a kernelized probabilistic model with an autoencoder and LSTM, later simplified to an LSTM-only variant to better handle lower-resolution data, and evaluates performance using and . The core contribution is a coordinated, interactive interface with a Comparison View and a Patterns View that helps researchers and grid operators interpret model behavior under diverse conditions and timeframes, thereby enhancing trust in forecasts. Case study insights demonstrate the tool’s ability to reveal when high-resolution data improves performance, identify diurnal-pattern dependencies, and guide deployment decisions, with plans to extend the platform to multiple models and economics-focused analyses for real-world applicability.

Abstract

Net load forecasting is crucial for energy planning and facilitating informed decision-making regarding trade and load distributions. However, evaluating forecasting models' performance against benchmark models remains challenging, thereby impeding experts' trust in the model's performance. In this context, there is a demand for technological interventions that allow scientists to compare models across various timeframes and solar penetration levels. This paper introduces a visual analytics-based application designed to compare the performance of deep-learning-based net load forecasting models with other models for probabilistic net load forecasting. This application employs carefully selected visual analytic interventions, enabling users to discern differences in model performance across different solar penetration levels, dataset resolutions, and hours of the day over multiple months. We also present observations made using our application through a case study, demonstrating the effectiveness of visualizations in aiding scientists in making informed decisions and enhancing trust in net load forecasting models.
Paper Structure (5 sections, 2 figures)

This paper contains 5 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Model description
  3. Visual analytics-based design
  4. Results from a Case Study
  5. Conclusion

Figures (2)

  • Figure 1: Visual analytic application (a) The Comparison View the facilitates comparison of CRPSS values between the net load forecasting model and the reference model at various data resolutions throughout the year. (b) The Patterns View aids in identifying performance trends across different hours of the day and months. (c), (d) and (e) denote filters for selecting different solar penetration levels, start and end dates, and specific months for the heatmap, respectively.
  • Figure 2: Results from a case study: (a), (b), (c) display CRPSS values at varying solar penetration levels, highlighting the model's superior performance with higher-resolution datasets. (d) Additionally, our application reveals insights such as the model's ability to learn and predict diurnal patterns, as evidenced by highlighted box-like patterns.