Leveraging Hypernetworks and Learnable Kernels for Consumer Energy Forecasting Across Diverse Consumer Types

TL;DR

HyperEnergy addresses cross-consumer energy forecasting by conditioning an LSTM predictor on parameters generated by a kernelized hypernetwork $H_k$ that outputs $Θ$ for the primary network. It integrates a Learnable Adaptive Kernel $K_{ ext{o}}$ composed of polynomial and RBF components with learnable reference points to transform inputs, feeding them through a Parameter Integration Module to supply $W_T,B_T$ to the LSTM. The approach achieves superior accuracy against 10 baselines on ten real-world datasets spanning student residences, detached homes, EV-charging homes, and townhouses, with ablations confirming contributions from both the hypernetwork and the learnable kernels. Practicality is demonstrated through reasonable training/inference times and strong cross-dataset performance, with potential for transfer learning across similar consumer types.

Abstract

Consumer energy forecasting is essential for managing energy consumption and planning, directly influencing operational efficiency, cost reduction, personalized energy management, and sustainability efforts. In recent years, deep learning techniques, especially LSTMs and transformers, have been greatly successful in the field of energy consumption forecasting. Nevertheless, these techniques have difficulties in capturing complex and sudden variations, and, moreover, they are commonly examined only on a specific type of consumer (e.g., only offices, only schools). Consequently, this paper proposes HyperEnergy, a consumer energy forecasting strategy that leverages hypernetworks for improved modeling of complex patterns applicable across a diversity of consumers. Hypernetwork is responsible for predicting the parameters of the primary prediction network, in our case LSTM. A learnable adaptable kernel, comprised of polynomial and radial basis function kernels, is incorporated to enhance performance. The proposed HyperEnergy was evaluated on diverse consumers including, student residences, detached homes, a home with electric vehicle charging, and a townhouse. Across all consumer types, HyperEnergy consistently outperformed 10 other techniques, including state-of-the-art models such as LSTM, AttentionLSTM, and transformer.

Figures (7)

• Figure 1: The proposed HyperEnergy, a deep learning technique, consists of three main components: (a) the kernelized hypernetwork, which contains learnable adaptive kernels, fully connected layers, and predicts weights and biases; (b) the parametric integration module, responsible for extracting and transforming weights and biases to ensure compatibility with LSTM; and (c) the primary network, consisting of LSTM and fully connected layers responsible for generating the final outputs.
• Figure 2: Student Residence 1: energy consumption characterized by observable seasonal variations affected by students' routines.
• Figure 4: Student Residence 1: actual versus the predicted value for top four models-- HyperEnergy, LSTM, AttentionLSTM, and XGBoost.
• Figure 6: House 1 energy consumption
• Figure 10: Ablation Study 1, Residence 2: HyperEnergy prediction with and without the learnable adaptive kernel compared to actual values.