Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding

Zixuan Liu, Siavash H. Khajavi, Guangkai Jiang

TL;DR

DetectiumFire addresses a critical gap in fire understanding by providing the first large-scale, multi-modal, publicly available dataset that combines real-world images and videos with synthetic data and rich text annotations. The dataset comprises 14.5k real-world fire images, 2.5k fire-related videos, 8k synthetic fire images, and 12k RLHF-derived preference pairs, all accompanied by bounding boxes and scene-level captions, enabling tasks from object detection to vision-language reasoning. Through comprehensive experiments, the authors demonstrate improved generalization over prior benchmarks, the value of diffusion-based synthetic data (via SFT and RLHF with Diffusion-DPO), and substantial gains in fire reasoning when finetuning VLMs on DetectiumFire. The resource is poised to advance safety-critical AI, synthetic data benchmarks, and open-vocabulary fire understanding, with broad applicability to disaster response and industrial monitoring.

Abstract

Recent advances in multi-modal models have demonstrated strong performance in tasks such as image generation and reasoning. However, applying these models to the fire domain remains challenging due to the lack of publicly available datasets with high-quality fire domain annotations. To address this gap, we introduce DetectiumFire, a large-scale, multi-modal dataset comprising of 22.5k high-resolution fire-related images and 2.5k real-world fire-related videos covering a wide range of fire types, environments, and risk levels. The data are annotated with both traditional computer vision labels (e.g., bounding boxes) and detailed textual prompts describing the scene, enabling applications such as synthetic data generation and fire risk reasoning. DetectiumFire offers clear advantages over existing benchmarks in scale, diversity, and data quality, significantly reducing redundancy and enhancing coverage of real-world scenarios. We validate the utility of DetectiumFire across multiple tasks, including object detection, diffusion-based image generation, and vision-language reasoning. Our results highlight the potential of this dataset to advance fire-related research and support the development of intelligent safety systems. We release DetectiumFire to promote broader exploration of fire understanding in the AI community. The dataset is available at https://kaggle.com/datasets/38b79c344bdfc55d1eed3d22fbaa9c31fad45e27edbbe9e3c529d6e5c4f93890

DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding

TL;DR

DetectiumFire addresses a critical gap in fire understanding by providing the first large-scale, multi-modal, publicly available dataset that combines real-world images and videos with synthetic data and rich text annotations. The dataset comprises 14.5k real-world fire images, 2.5k fire-related videos, 8k synthetic fire images, and 12k RLHF-derived preference pairs, all accompanied by bounding boxes and scene-level captions, enabling tasks from object detection to vision-language reasoning. Through comprehensive experiments, the authors demonstrate improved generalization over prior benchmarks, the value of diffusion-based synthetic data (via SFT and RLHF with Diffusion-DPO), and substantial gains in fire reasoning when finetuning VLMs on DetectiumFire. The resource is poised to advance safety-critical AI, synthetic data benchmarks, and open-vocabulary fire understanding, with broad applicability to disaster response and industrial monitoring.

Abstract

Recent advances in multi-modal models have demonstrated strong performance in tasks such as image generation and reasoning. However, applying these models to the fire domain remains challenging due to the lack of publicly available datasets with high-quality fire domain annotations. To address this gap, we introduce DetectiumFire, a large-scale, multi-modal dataset comprising of 22.5k high-resolution fire-related images and 2.5k real-world fire-related videos covering a wide range of fire types, environments, and risk levels. The data are annotated with both traditional computer vision labels (e.g., bounding boxes) and detailed textual prompts describing the scene, enabling applications such as synthetic data generation and fire risk reasoning. DetectiumFire offers clear advantages over existing benchmarks in scale, diversity, and data quality, significantly reducing redundancy and enhancing coverage of real-world scenarios. We validate the utility of DetectiumFire across multiple tasks, including object detection, diffusion-based image generation, and vision-language reasoning. Our results highlight the potential of this dataset to advance fire-related research and support the development of intelligent safety systems. We release DetectiumFire to promote broader exploration of fire understanding in the AI community. The dataset is available at https://kaggle.com/datasets/38b79c344bdfc55d1eed3d22fbaa9c31fad45e27edbbe9e3c529d6e5c4f93890

Paper Structure

This paper contains 33 sections, 2 equations, 4 figures, 20 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. DetectiumFire Dataset
  4. Real-World Dataset
  5. Synthetic Dataset
  6. Experiments
  7. Experimental Details
  8. Experiment Results
  9. Conclusion
  10. Acknowledgments
  11. Discussions
  12. Potential Use Cases
  13. Benchmark for Synthetic Fire Data Generation
  14. Fire Video Generation
  15. Advanced Fire Reasoning with Vision-Language Models and AI Agents
  16. ...and 18 more sections

Figures (4)

  • Figure 1: Screenshot of the fire annotation tool. The tool displays each image alongside the original GPT-generated caption and allows human annotators to edit and finalize a concise, domain-specific prompt.
  • Figure 2: Representative samples from DetectiumFire showcasing diverse fire scenarios across both indoor and outdoor settings.
  • Figure 3: Additional example videos from the DetectiumFire dataset showcase various fire scenes.
  • Figure 4: Examples of non-fire images that commonly trigger false positives in current fire detection models.