Physics-Constrained Cross-Resolution Enhancement Network for Optics-Guided Thermal UAV Image Super-Resolution

TL;DR

This paper tackles the challenge of generating high-quality thermal UAV super-resolution images guided by optical data, addressing the loss of optical high-frequency details and thermophysical inconsistencies in prior cross-modal methods. It introduces PCNet, a dual-branch network that enables cross-resolution mutual enhancement (CRME) while enforcing physics-based guidance through a PDTM and a Temperature Consistency Loss (TCLoss). The key contributions are the CRME for bidirectional, resolution-aware feature exchange, the PDTM that simulates heat diffusion to constrain optical guidance, and the TCLoss that enforces region-wise temperature consistency and boundary smoothness. Extensive experiments on VGTSR2.0 and DroneVehicle show PCNet outperforms state-of-the-art SISR and GISR approaches in reconstruction quality and downstream tasks such as semantic segmentation and object detection, highlighting its practical impact for robust all-weather UAV sensing.

Abstract

Optics-guided thermal UAV image super-resolution has attracted significant research interest due to its potential in all-weather monitoring applications. However, existing methods typically compress optical features to match thermal feature dimensions for cross-modal alignment and fusion, which not only causes the loss of high-frequency information that is beneficial for thermal super-resolution, but also introduces physically inconsistent artifacts such as texture distortions and edge blurring by overlooking differences in the imaging physics between modalities. To address these challenges, we propose PCNet to achieve cross-resolution mutual enhancement between optical and thermal modalities, while physically constraining the optical guidance process via thermal conduction to enable robust thermal UAV image super-resolution. In particular, we design a Cross-Resolution Mutual Enhancement Module (CRME) to jointly optimize thermal image super-resolution and optical-to-thermal modality conversion, facilitating effective bidirectional feature interaction across resolutions while preserving high-frequency optical priors. Moreover, we propose a Physics-Driven Thermal Conduction Module (PDTM) that incorporates two-dimensional heat conduction into optical guidance, modeling spatially-varying heat conduction properties to prevent inconsistent artifacts. In addition, we introduce a temperature consistency loss that enforces regional distribution consistency and boundary gradient smoothness to ensure generated thermal images align with real-world thermal radiation principles. Extensive experiments on VGTSR2.0 and DroneVehicle datasets demonstrate that PCNet significantly outperforms state-of-the-art methods on both reconstruction quality and downstream tasks including semantic segmentation and object detection.

Figures (15)

• Figure 1: Inaccurate guidance due to the loss of optical high-frequency information and cross-modal imaging discrepancies. We compare SISR methods such as Restormer with GISR methods including UGSR, MGNet, CENet, and GDNet. Our proposed PCNet preserves high-frequency details from optical priors, providing effective guidance for thermal image super-resolution while maintaining high thermophysical consistency between the generated results and the Ground Truth (GT).
• Figure 2: The overall structure of the proposed PCNet. The Cross-Resolution Mutual Enhancement Module (CRME) employs cross-resolution cross-attention to achieve mutual enhancement between modalities while preserving high-frequency optical priors and jointly optimizing SR and MC tasks. The Physics-Driven Thermal Conduction Module (PDTM) achieves physically consistent optical guidance and thermal information enhancement by simulating how thermal images respond to heat diffusion at the edges of real-world objects. Additionally, we incorporate a physical consistency constraint into the loss function to further reduce the discrepancy between SR thermal images and HR thermal images.
• Figure 3: Framework of the proposed CRME, illustrating the SR branch (Left) and the MC branch (Right). Optical and thermal features are aligned via SPL and interact through MGL to mitigate domain gaps and resolution discrepancies.
• Figure 4: The network structure of HTL. Among these, spatial self-correlation (S-SC) and channel self-correlation (C-SC) aim to efficiently aggregate hierarchical information with computational complexity that scales linearly with window size.
• Figure 5: Qualitative comparison of super-resolution methods at $\times$4 SR with BI and BD degradation models in the VGTSR2.0 dataset. Experimental results demonstrate that our proposed PCNet effectively utilizes information from optical images to recover texture details and clear edges closer to the GT.