Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solar-VLM: Multimodal Vision-Language Models for Augmented Solar Power Forecasting

Hang Fan, Haoran Pei, Runze Liang, Weican Liu, Long Cheng, Wei Wei

Abstract

Photovoltaic (PV) power forecasting plays a critical role in power system dispatch and market participation. Because PV generation is highly sensitive to weather conditions and cloud motion, accurate forecasting requires effective modeling of complex spatiotemporal dependencies across multiple information sources. Although recent studies have advanced AI-based forecasting methods, most fail to fuse temporal observations, satellite imagery, and textual weather information in a unified framework. This paper proposes Solar-VLM, a large-language-model-driven framework for multimodal PV power forecasting. First, modality-specific encoders are developed to extract complementary features from heterogeneous inputs. The time-series encoder adopts a patch-based design to capture temporal patterns from multivariate observations at each site. The visual encoder, built upon a Qwen-based vision backbone, extracts cloud-cover information from satellite images. The text encoder distills historical weather characteristics from textual descriptions. Second, to capture spatial dependencies across geographically distributed PV stations, a cross-site feature fusion mechanism is introduced. Specifically, a Graph Learner models inter-station correlations through a graph attention network constructed over a K-nearest-neighbor (KNN) graph, while a cross-site attention module further facilitates adaptive information exchange among sites. Finally, experiments conducted on data from eight PV stations in a northern province of China demonstrate the effectiveness of the proposed framework. Our proposed model is publicly available at https://github.com/rhp413/Solar-VLM.

Solar-VLM: Multimodal Vision-Language Models for Augmented Solar Power Forecasting

Abstract

Photovoltaic (PV) power forecasting plays a critical role in power system dispatch and market participation. Because PV generation is highly sensitive to weather conditions and cloud motion, accurate forecasting requires effective modeling of complex spatiotemporal dependencies across multiple information sources. Although recent studies have advanced AI-based forecasting methods, most fail to fuse temporal observations, satellite imagery, and textual weather information in a unified framework. This paper proposes Solar-VLM, a large-language-model-driven framework for multimodal PV power forecasting. First, modality-specific encoders are developed to extract complementary features from heterogeneous inputs. The time-series encoder adopts a patch-based design to capture temporal patterns from multivariate observations at each site. The visual encoder, built upon a Qwen-based vision backbone, extracts cloud-cover information from satellite images. The text encoder distills historical weather characteristics from textual descriptions. Second, to capture spatial dependencies across geographically distributed PV stations, a cross-site feature fusion mechanism is introduced. Specifically, a Graph Learner models inter-station correlations through a graph attention network constructed over a K-nearest-neighbor (KNN) graph, while a cross-site attention module further facilitates adaptive information exchange among sites. Finally, experiments conducted on data from eight PV stations in a northern province of China demonstrate the effectiveness of the proposed framework. Our proposed model is publicly available at https://github.com/rhp413/Solar-VLM.

Paper Structure

This paper contains 30 sections, 40 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background and motivation
  3. Traditional Prediction Methods
  4. Deep Learning-based Prediction Methods
  5. Time Series–based PV Forecasting
  6. Spatio-Temporal Forecasting across Multiple PV Sites
  7. Vision-driven and Multimodal Forecasting
  8. Contributions
  9. Problem Formulation
  10. Methodology
  11. Single-Modality Encoders
  12. Time Series Encoder
  13. Text Encoder
  14. Visual Encoder
  15. Multimodal Feature Fusion
  16. ...and 15 more sections

Figures (7)

  • Figure 1: Overview of the proposed model.
  • Figure 2: Overview of the proposed two-stage cross-site joint modeling strategy.
  • Figure 3: MAE boxplots for different models at different forecasting horizons.
  • Figure 4: PV power prediction curves of compared models against the ground truth on the test set at different forecasting horizons.
  • Figure 5: Forecasting results of different ablation variants on a representative cloudy day.
  • ...and 2 more figures