Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Target Concept Tuning Improves Extreme Weather Forecasting

Shijie Ren, Xinyue Gu, Ziheng Peng, Haifan Zhang, Peisong Niu, Bo Wu, Xiting Wang, Liang Sun, Jirong Wen

Abstract

Deep learning models for meteorological forecasting often fail in rare but high-impact events such as typhoons, where relevant data is scarce. Existing fine-tuning methods typically face a trade-off between overlooking these extreme events and overfitting them at the expense of overall performance. We propose TaCT, an interpretable concept-gated fine-tuning framework that solves the aforementioned issue by selective model improvement: models are adapted specifically for failure cases while preserving performance in common scenarios. To this end, TaCT automatically discovers failure-related internal concepts using Sparse Autoencoders and counterfactual analysis, and updates parameters only when the corresponding concepts are activated, rather than applying uniform adaptation. Experiments show consistent improvements in typhoon forecasting across different regions without degrading other meteorological variables. The identified concepts correspond to physically meaningful circulation patterns, revealing model biases and supporting trustworthy adaptation in scientific forecasting tasks. The code is available at https://anonymous.4open.science/r/Concept-Gated-Fine-tune-62AC.

Target Concept Tuning Improves Extreme Weather Forecasting

Abstract

Deep learning models for meteorological forecasting often fail in rare but high-impact events such as typhoons, where relevant data is scarce. Existing fine-tuning methods typically face a trade-off between overlooking these extreme events and overfitting them at the expense of overall performance. We propose TaCT, an interpretable concept-gated fine-tuning framework that solves the aforementioned issue by selective model improvement: models are adapted specifically for failure cases while preserving performance in common scenarios. To this end, TaCT automatically discovers failure-related internal concepts using Sparse Autoencoders and counterfactual analysis, and updates parameters only when the corresponding concepts are activated, rather than applying uniform adaptation. Experiments show consistent improvements in typhoon forecasting across different regions without degrading other meteorological variables. The identified concepts correspond to physically meaningful circulation patterns, revealing model biases and supporting trustworthy adaptation in scientific forecasting tasks. The code is available at https://anonymous.4open.science/r/Concept-Gated-Fine-tune-62AC.
Paper Structure (32 sections, 22 equations, 10 figures, 3 tables)

This paper contains 32 sections, 22 equations, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Methods
  4. Counterfactual Concept Localization
  5. Unsupervised Concept Decomposition
  6. Continuous Counterfactual Reasoning
  7. Concept-Gated Fine-tuning
  8. Concept-Gated Fine-tuning
  9. Loss Function
  10. Method Comparison
  11. Experiments
  12. Experimental Settings
  13. Overall Performance
  14. Ablation Study and Sensitivity Analysis
  15. General Capabilities
  16. ...and 17 more sections

Figures (10)

  • Figure 1: The overall framework of TaCT, comprising two main modules: (1) counterfactual concept localization, which decomposes input hidden representations and identifies key concepts through continuous counterfactual reasoning; (2) concept-gated fine-tuning, which selectively fine-tunes activated concepts without affecting others.
  • Figure 2: 72-Hours Performance. WP denotes the Western Pacific. “Typhoon pressure” refers to the minimum sea-level pressure in Pa, and “Typhoon wind speed” represents the maximum wind speed in m/s.
  • Figure 3: Sensitivity Analysis. (a, b) and (c,d) present the 6-hour forecast results of minimum sea level pressure (MSL) and maximum wind speed of typhoons across different regions under varying parameter sizes. (e) presents the 6-hour MSL forecast results under different model activation threshold settings. (f) presents the MSL forecast results under different concept number settings.
  • Figure 4: The impact of different fine-tuning methods on general capabilities. Where 0 represents the performance of the base model, and the values indicate the change of different methods relative to the original model, with lower values being better.
  • Figure 5: Six concepts learned by SAE.
  • ...and 5 more figures