Zero-shot Load Forecasting for Integrated Energy Systems: A Large Language Model-based Framework with Multi-task Learning

TL;DR

This work tackles load forecasting in integrated energy systems under zero-shot conditions, where traditional data-driven methods struggle with transferability and data scarcity. It introduces TSLLM-Load Forecasting Mechanism, a three-component framework that preprocesses multi-source data, converts time-series into LLM-friendly prompts via a time-series multi-task prompt generation, and uses a pre-trained LLM with a reconstruction layer to forecast future loads. The key contributions include a reversible normalization-based preprocessing pipeline, a seasonal-trend decomposition and patch-based prompt construction, a multi-task embedding and cosine-similarity alignment scheme, and a selective training strategy that preserves linguistic knowledge while adapting to time-series tasks. Empirical results on 20 Australian solar households show superior conventional and zero-shot performance, with notable improvements over strong baselines and robust transferability across households, highlighting the method’s potential for renewable integration and smart grid applications.

Abstract

The growing penetration of renewable energy sources in power systems has increased the complexity and uncertainty of load forecasting, especially for integrated energy systems with multiple energy carriers. Traditional forecasting methods heavily rely on historical data and exhibit limited transferability across different scenarios, posing significant challenges for emerging applications in smart grids and energy internet. This paper proposes the TSLLM-Load Forecasting Mechanism, a novel zero-shot load forecasting framework based on large language models (LLMs) to address these challenges. The framework consists of three key components: a data preprocessing module that handles multi-source energy load data, a time series prompt generation module that bridges the semantic gap between energy data and LLMs through multi-task learning and similarity alignment, and a prediction module that leverages pre-trained LLMs for accurate forecasting. The framework's effectiveness was validated on a real-world dataset comprising load profiles from 20 Australian solar-powered households, demonstrating superior performance in both conventional and zero-shot scenarios. In conventional testing, our method achieved a Mean Squared Error (MSE) of 0.4163 and a Mean Absolute Error (MAE) of 0.3760, outperforming existing approaches by at least 8\%. In zero-shot prediction experiments across 19 households, the framework maintained consistent accuracy with a total MSE of 11.2712 and MAE of 7.6709, showing at least 12\% improvement over current methods. The results validate the framework's potential for accurate and transferable load forecasting in integrated energy systems, particularly beneficial for renewable energy integration and smart grid applications.

Figures (9)

• Figure 1: Comparison of load forecasting schemes: (a) Traditional machine learning methods and (b) Large language model-based approach. The traditional method requires extensive historical data and exhibits limited transferability, while the LLM-based method enables zero-shot prediction across different data features.
• Figure 2: The workflow of TSLLM-Load Forecasting Mechanism, illustrating the systematic integration of data preprocessing, prompt generation, and prediction modules.
• Figure 3: Training process flowchart of the TSLLM-Load Forecasting Mechanism, illustrating parameter optimization and data flow through different components.
• Figure 4: Experimental workflow diagram showing the input processing, prediction generation, and output formation stages of the TSLLM-Load Forecasting Mechanism.
• Figure 5: Input-output structure and prompt template design of the TSLLM-Load Forecasting Mechanism, showing the transformation from raw data to prediction output.