Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning

Suhasnadh Reddy Veluru, Sai Teja Erukude, Viswa Chaitanya Marella

TL;DR

The Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE, with performance above any model in both accuracy and robustness, which underscores the growing potential of attention-based architectures in accurately capturing complex temporal patterns in power consumption data.

Abstract

Accurate short-term power load forecasting is important to effectively manage, optimize, and ensure the robustness of modern power systems. This paper performs an empirical evaluation of a traditional statistical model and deep learning approaches for predicting short-term energy load. Four models, namely ARIMA, LSTM, BiLSTM, and Transformer, were leveraged on the PJM Hourly Energy Consumption data. The data processing involved interpolation, normalization, and a sliding-window sequence method. Each model's forecasting performance was evaluated for the 24-hour horizon using MAE, RMSE, and MAPE. Of the models tested, the Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE, with performance above any model in both accuracy and robustness. These findings underscore the growing potential of attention-based architectures in accurately capturing complex temporal patterns in power consumption data.

From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning

TL;DR

The Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE, with performance above any model in both accuracy and robustness, which underscores the growing potential of attention-based architectures in accurately capturing complex temporal patterns in power consumption data.

Abstract

Accurate short-term power load forecasting is important to effectively manage, optimize, and ensure the robustness of modern power systems. This paper performs an empirical evaluation of a traditional statistical model and deep learning approaches for predicting short-term energy load. Four models, namely ARIMA, LSTM, BiLSTM, and Transformer, were leveraged on the PJM Hourly Energy Consumption data. The data processing involved interpolation, normalization, and a sliding-window sequence method. Each model's forecasting performance was evaluated for the 24-hour horizon using MAE, RMSE, and MAPE. Of the models tested, the Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE, with performance above any model in both accuracy and robustness. These findings underscore the growing potential of attention-based architectures in accurately capturing complex temporal patterns in power consumption data.
Paper Structure (13 sections, 6 figures, 1 table)

This paper contains 13 sections, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data and Preprocessing
  4. Forecasting Models
  5. ARIMA
  6. LSTM and BiLSTM
  7. Transformer
  8. Training Procedure
  9. Results and Discussion
  10. Quantitative Performance
  11. Analysis of Model Behavior
  12. Contextual Comparison with Literature
  13. Conclusion and Future Work

Figures (6)

  • Figure 1: Energy Consumption Forecasting: ARIMA vs True Values for One Week. The plot compares ARIMA model predictions (red dashed line) with the actual energy consumption (black line) over 7 days.
  • Figure 2: Energy Consumption Forecasting: LSTM vs. True Values for One Week. The plot shows LSTM model predictions (blue dashed line) compared to actual energy usage (black line) over a representative week.
  • Figure 3: Energy Consumption Forecasting: BiLSTM vs True Values for One Week. The plot depicts BiLSTM model predictions (green dashed line) against the true energy consumption (black line) over 168 hours.
  • Figure 4: Energy Consumption Forecasting: Transformer vs True Values for One Week. The Transformer model prediction (purple dashed line) closely follows actual energy consumption (black line), showing improved accuracy and robustness.
  • Figure 5: Comparison of MAE and RMSE across Models
  • ...and 1 more figures