Learning to Calibrate for Reliable Visual Fire Detection

TL;DR

This work tackles the tendency of deep visual fire detectors to be overconfident by introducing a differentiable Expected Calibration Error (ECE) loss and online calibration guided by curriculum learning. The approach couples the differentiable ECE loss with cross-entropy, and dynamically weights the calibration term via a schedule $L = L_n + \frac{c_e-s_e}{N-s_e} \times \gamma_E \times L_e$, thereby gradually improving uncertainty calibration as training progresses. Key contributions include (1) a differentiable ECE loss for multi-class fire detection, (2) a curriculum-based strategy to balance accuracy and reliability during training, and (3) extensive validation on the DFAN and EdgeFireSmoke datasets showing improved calibration (lower ECE) with minimal drops in accuracy. The results demonstrate that calibration-aware training yields more reliable decisions in safety-critical fire detection tasks, with practical implications for deploying vision-based fire detection systems in real-world settings.

Abstract

Fire is characterized by its sudden onset and destructive power, making early fire detection crucial for ensuring human safety and protecting property. With the advancement of deep learning, the application of computer vision in fire detection has significantly improved. However, deep learning models often exhibit a tendency toward overconfidence, and most existing works focus primarily on enhancing classification performance, with limited attention given to uncertainty modeling. To address this issue, we propose transforming the Expected Calibration Error (ECE), a metric for measuring uncertainty, into a differentiable ECE loss function. This loss is then combined with the cross-entropy loss to guide the training process of multi-class fire detection models. Additionally, to achieve a good balance between classification accuracy and reliable decision, we introduce a curriculum learning-based approach that dynamically adjusts the weight of the ECE loss during training. Extensive experiments are conducted on two widely used multi-class fire detection datasets, DFAN and EdgeFireSmoke, validating the effectiveness of our uncertainty modeling method.

Figures (10)

• Figure 1: Non-fire images with interfering objects.
• Figure 2: Reliability diagram under perfect calibration.
• Figure 3: Two indistinguishable reliability diagrams.
• Figure 4: Curve of sigmoid function.
• Figure 5: Curve of accuracy with predicted probability changing from zero to one.