Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Meteorology-Driven GPT4AP: A Multi-Task Forecasting LLM for Atmospheric Air Pollution in Data-Scarce Settings

Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-Phelan

Abstract

Accurate forecasting of air pollution is important for environmental monitoring and policy support, yet data-driven models often suffer from limited generalization in regions with sparse observations. This paper presents Meteorology-Driven GPT for Air Pollution (GPT4AP), a parameter-efficient multi-task forecasting framework based on a pre-trained GPT-2 backbone and Gaussian rank-stabilized low-rank adaptation (rsLoRA). The model freezes the self-attention and feed-forward layers and adapts lightweight positional and output modules, substantially reducing the number of trainable parameters. GPT4AP is evaluated on six real-world air quality monitoring datasets under few-shot, zero-shot, and long-term forecasting settings. In the few-shot regime using 10% of the training data, GPT4AP achieves an average MSE/MAE of 0.686/0.442, outperforming DLinear (0.728/0.530) and ETSformer (0.734/0.505). In zero-shot cross-station transfer, the proposed model attains an average MSE/MAE of 0.529/0.403, demonstrating improved generalization compared with existing baselines. In long-term forecasting with full training data, GPT4AP remains competitive, achieving an average MAE of 0.429, while specialized time-series models show slightly lower errors. These results indicate that GPT4AP provides a data-efficient forecasting approach that performs robustly under limited supervision and domain shift, while maintaining competitive accuracy in data-rich settings.

Meteorology-Driven GPT4AP: A Multi-Task Forecasting LLM for Atmospheric Air Pollution in Data-Scarce Settings

Abstract

Accurate forecasting of air pollution is important for environmental monitoring and policy support, yet data-driven models often suffer from limited generalization in regions with sparse observations. This paper presents Meteorology-Driven GPT for Air Pollution (GPT4AP), a parameter-efficient multi-task forecasting framework based on a pre-trained GPT-2 backbone and Gaussian rank-stabilized low-rank adaptation (rsLoRA). The model freezes the self-attention and feed-forward layers and adapts lightweight positional and output modules, substantially reducing the number of trainable parameters. GPT4AP is evaluated on six real-world air quality monitoring datasets under few-shot, zero-shot, and long-term forecasting settings. In the few-shot regime using 10% of the training data, GPT4AP achieves an average MSE/MAE of 0.686/0.442, outperforming DLinear (0.728/0.530) and ETSformer (0.734/0.505). In zero-shot cross-station transfer, the proposed model attains an average MSE/MAE of 0.529/0.403, demonstrating improved generalization compared with existing baselines. In long-term forecasting with full training data, GPT4AP remains competitive, achieving an average MAE of 0.429, while specialized time-series models show slightly lower errors. These results indicate that GPT4AP provides a data-efficient forecasting approach that performs robustly under limited supervision and domain shift, while maintaining competitive accuracy in data-rich settings.

Paper Structure

This paper contains 21 sections, 14 equations, 2 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Air Pollution Prediction Problem
  5. Overview of GPT4AP
  6. Input Representation and Temporal Tokenization
  7. Trainable Positional Encoding
  8. Frozen GPT-2 Backbone
  9. Gaussian Rank-Stabilized LoRA (rsLoRA)
  10. Prediction Head for Multi-Horizon Forecasting
  11. Training Protocols for Long-Term, Few-Shot, and Zero-Shot Forecasting
  12. Experimental Setup
  13. Dataset Description
  14. Baseline Models and Implementation Details
  15. Experimental Results
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Overview of GPT4AP. Multivariate air-pollution and meteorological time series are embedded and projected into temporal tokens. Trainable positional encoding is added before feeding tokens into a frozen GPT-2 backbone. Parameter-efficient Gaussian rsLoRA is applied to lightweight adaptation modules, while the pretrained transformer weights remain frozen. A prediction head produces outputs for few-shot, long-term, and zero-shot forecasting across multiple horizons.
  • Figure 2: Rank ablation study of Gaussian rsLoRA. (a) Forecasting error (MSE) decreases as rank increases and saturates beyond $r{=}32$. (b) Percentage of trainable parameters grows approximately linearly with rank. The best trade-off is achieved at $r{=}32$, which attains near-peak performance (99.9% of the minimum MSE on average) while training only 0.309% of the full model parameters.