ExEBench: Benchmarking Foundation Models on Extreme Earth Events
Shan Zhao, Zhitong Xiong, Jie Zhao, Xiao Xiang Zhu
TL;DR
ExEBench addresses the need for reliable evaluation of foundation models on extreme Earth events by curating a diverse, multi-modal benchmark spanning seven event categories and designing tasks that reflect real-world disaster management needs. The framework analyzes FM generalizability across data modalities (vision, EO, weather & climate) and fine-tuning strategies, including full training, decoder-only tuning, and LoRA, across variable spatial, temporal, and spectral scales. Key findings show that pre-trained weights accelerate convergence and improve performance when modalities align, yet temporal dynamics and cross-domain transfer remain challenging, underscoring the importance of data alignment and spectral/physical properties in model design. The work provides a practical platform for advancing robust FM-based disaster management tools and outlines directions for richer temporal modeling, sensor-aware architectures, and expanded modality fusion to better capture the cascading effects of extreme events under climate change.
Abstract
Our planet is facing increasingly frequent extreme events, which pose major risks to human lives and ecosystems. Recent advances in machine learning (ML), especially with foundation models (FMs) trained on extensive datasets, excel in extracting features and show promise in disaster management. Nevertheless, these models often inherit biases from training data, challenging their performance over extreme values. To explore the reliability of FM in the context of extreme events, we introduce \textbf{ExE}Bench (\textbf{Ex}treme \textbf{E}arth Benchmark), a collection of seven extreme event categories across floods, wildfires, storms, tropical cyclones, extreme precipitation, heatwaves, and cold waves. The dataset features global coverage, varying data volumes, and diverse data sources with different spatial, temporal, and spectral characteristics. To broaden the real-world impact of FMs, we include multiple challenging ML tasks that are closely aligned with operational needs in extreme events detection, monitoring, and forecasting. ExEBench aims to (1) assess FM generalizability across diverse, high-impact tasks and domains, (2) promote the development of novel ML methods that benefit disaster management, and (3) offer a platform for analyzing the interactions and cascading effects of extreme events to advance our understanding of Earth system, especially under the climate change expected in the decades to come. The dataset and code are public https://github.com/zhaoshan2/EarthExtreme-Bench.