DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response

TL;DR

DisasterM3 tackles the lack of disaster-focused vision–language resources by assembling a global, multi-hazard, multi-sensor dataset with bi-temporal optical and SAR imagery and 9 disaster-oriented tasks. The work benchmarks 14 VLMs, revealing domain gaps that hinder disaster analysis and showing that fine-tuning with disaster-specific data yields robust improvements and better cross-sensor generalization. By providing both Instruct and Bench data splits and extensive baselines, DisasterM3 establishes a practical foundation for advancing AI-assisted disaster assessment and response in Earth observation contexts. The dataset, code, and baselines enable researchers and practitioners to develop more capable, interpretable, and deployable disaster-focused VLMs.

Abstract

Large vision-language models (VLMs) have made great achievements in Earth vision. However, complex disaster scenes with diverse disaster types, geographic regions, and satellite sensors have posed new challenges for VLM applications. To fill this gap, we curate a remote sensing vision-language dataset (DisasterM3) for global-scale disaster assessment and response. DisasterM3 includes 26,988 bi-temporal satellite images and 123k instruction pairs across 5 continents, with three characteristics: 1) Multi-hazard: DisasterM3 involves 36 historical disaster events with significant impacts, which are categorized into 10 common natural and man-made disasters. 2)Multi-sensor: Extreme weather during disasters often hinders optical sensor imaging, making it necessary to combine Synthetic Aperture Radar (SAR) imagery for post-disaster scenes. 3) Multi-task: Based on real-world scenarios, DisasterM3 includes 9 disaster-related visual perception and reasoning tasks, harnessing the full potential of VLM's reasoning ability with progressing from disaster-bearing body recognition to structural damage assessment and object relational reasoning, culminating in the generation of long-form disaster reports. We extensively evaluated 14 generic and remote sensing VLMs on our benchmark, revealing that state-of-the-art models struggle with the disaster tasks, largely due to the lack of a disaster-specific corpus, cross-sensor gap, and damage object counting insensitivity. Focusing on these issues, we fine-tune four VLMs using our dataset and achieve stable improvements across all tasks, with robust cross-sensor and cross-disaster generalization capabilities. The code and data are available at: https://github.com/Junjue-Wang/DisasterM3.

Figures (13)

• Figure 1: Task taxonomy in DisasterM3 dataset. Each scene includes the paired pre- and post-disaster images. The modalities of post-disaster images are optical or SAR. The 9 tasks derive from 5 essential capabilities for bi-temporal disaster assessment and response: recognition, counting, localization, reasoning, and report generation.
• Figure 2: The DisasterM3 dataset involves 36 significant natural and man-made disaster events (10 types) across five continents. Diverse disaster-centric tasks provide a comprehensive evaluation benchmark for VLMs.
• Figure 3: Disaster referring segmentation task involves disaster-bearing body mapping and risk analysis. By querying, rescuers could accurately locate the disaster-related objects.
• Figure 4: Damaged object relational reasoning task describes spatial relationships between key facilities, revealing crucial patterns in the object dependencies.
• Figure 5: (Left) The disaster comprehensive reports provide a holistic analysis of disaster situations and evidence-based rescue support. It is notable that immediate earthquake response prioritizes deploying temporary shelters within the stadium for displaced survivors, an intervention demonstrated in the post-disaster image. (Right) Word cloud of reports shows that the disaster-centric words have a considerable degree of diversity.