HATIR: Heat-Aware Diffusion for Turbulent Infrared Video Super-Resolution

TL;DR

This work tackles infrared video super-resolution under atmospheric turbulence, where modality gaps and nonuniform distortions complicate restoration. It introduces HATIR, a diffusion-based framework that jointly models turbulent degradation and detail loss using heat-aware priors, incorporating a Phasor-Guided Flow Estimator and a Turbulence-Aware Decoder to unify alignment and restoration. Key innovations include the PhasorMask and Frequency-Weighted Attention for robust motion guidance, a physics-informed heat-aware denoising path, and the FLIR-IVSR dataset with 640 LR-HR sequences to benchmark turbulent infrared VSR. The approach achieves state-of-the-art results across fidelity and perceptual metrics, demonstrating robustness to severe turbulence and infrared-specific degradation with practical implications for safety-critical tasks in low-visibility conditions.

Abstract

Infrared video has been of great interest in visual tasks under challenging environments, but often suffers from severe atmospheric turbulence and compression degradation. Existing video super-resolution (VSR) methods either neglect the inherent modality gap between infrared and visible images or fail to restore turbulence-induced distortions. Directly cascading turbulence mitigation (TM) algorithms with VSR methods leads to error propagation and accumulation due to the decoupled modeling of degradation between turbulence and resolution. We introduce HATIR, a Heat-Aware Diffusion for Turbulent InfraRed Video Super-Resolution, which injects heat-aware deformation priors into the diffusion sampling path to jointly model the inverse process of turbulent degradation and structural detail loss. Specifically, HATIR constructs a Phasor-Guided Flow Estimator, rooted in the physical principle that thermally active regions exhibit consistent phasor responses over time, enabling reliable turbulence-aware flow to guide the reverse diffusion process. To ensure the fidelity of structural recovery under nonuniform distortions, a Turbulence-Aware Decoder is proposed to selectively suppress unstable temporal cues and enhance edge-aware feature aggregation via turbulence gating and structure-aware attention. We built FLIR-IVSR, the first dataset for turbulent infrared VSR, comprising paired LR-HR sequences from a FLIR T1050sc camera (1024 X 768) spanning 640 diverse scenes with varying camera and object motion conditions. This encourages future research in infrared VSR. Project page: https://github.com/JZ0606/HATIR

Figures (7)

• Figure 1: Infrared VSR performance under turbulence conditions evaluated by HATIR on the proposed FLIR-IVSR dataset. The graph illustrates grayscale fluctuations along the orange-marked sampling line over time (30 video frames).
• Figure 2: Given a low-resolution (LR) turbulent infrared video sequence $\mathbf{I}_{LR} = \{\mathbf{I}_1, \mathbf{I}_2, \dots, \mathbf{I}_N\}$, HATIR reconstructs a high-resolution (HR) sequence $\mathbf{I}_{HR} = \{\hat{\mathbf{I}}_1, \hat{\mathbf{I}}_2, \dots, \hat{\mathbf{I}}_N\}$ with suppressed turbulence distortions and enhanced temporal coherence. The proposed unified latent diffusion framework jointly addresses spatial degradation removal and inter-frame alignment for infrared videos under atmospheric turbulence.
• Figure 3: Overview of PhasorFlow.
• Figure 4: Qualitative results. The first row is from the static scenes of the $\mathrm{M^3FD}$ dataset, while the second and third rows are from the FLIR-IVSR dataset. MambaTM, DATUM, and Turb-Seg are combined with BasicVSR to form a two-stage pipeline.
• Figure 5: Qualitative ablation on the PhasorFlow.