TDiff: Thermal Plug-And-Play Prior with Patch-Based Diffusion

TL;DR

This work tackles thermal image restoration under data scarcity and structured noise by introducing TDiff, a patch-based diffusion framework. By training on patches of 64x64 and 128x128, it learns localized priors that handle edge-preserving denoising, super-resolution, and deblurring. Full-resolution results are obtained by denoising overlapping patches and blending them with smooth windows while enforcing data consistency during diffusion. On the FLIR dataset and real cameras, TDiff delivers higher structural similarity and competitive PSNR compared to strong baselines, illustrating a unified restoration pipeline for thermal images. The approach offers data-efficient restoration useful for low-cost thermal cameras and could extend to other modalities.

Abstract

Thermal images from low-cost cameras often suffer from low resolution, fixed pattern noise, and other localized degradations. Available datasets for thermal imaging are also limited in both size and diversity. To address these challenges, we propose a patch-based diffusion framework (TDiff) that leverages the local nature of these distortions by training on small thermal patches. In this approach, full-resolution images are restored by denoising overlapping patches and blending them using smooth spatial windowing. To our knowledge, this is the first patch-based diffusion framework that models a learned prior for thermal image restoration across multiple tasks. Experiments on denoising, super-resolution, and deblurring demonstrate strong results on both simulated and real thermal data, establishing our method as a unified restoration pipeline.

Figures (6)

• Figure 1: Overview of TDiff pipeline. (Top) A degraded thermal image $x_t$ is divided into overlapping patches, denoised by our patch-based diffusion model, and recombined into $\hat{x}_t$, which guides $x_{t+1}$ via simulated degradation $y_t$. The right panel shows patch division, diffusion restoration, and stitching. (Bottom) TDiff performs 2$\times$ and 4$\times$ super-resolution, denoising, and deblurring, with inputs (left) and outputs (right) showing cleaner detail and preserved thermal boundaries.
• Figure 2: Simulated FLIR results with added FPN. TDiff outperforms baselines for (a) denoising, (b) 2$\times$ (top) and 4$\times$ (bottom) super-resolution, and (c) deblurring, preserving edges, suppressing noise, and maintaining thermal gradients.
• Figure 3: Impact of patch size and overlap on restoration quality. (a, b) Smaller patch sizes reduce unconditional sample quality. (c) Non-overlapping reconstruction leads to visible seams. (d) TDiff variants show trade-offs between patch size and restoration quality.
• Figure 4: Real-world thermal image restoration results, comparing our method to competing approaches. (a) Shows denoising results on FLIR Boson+ data, (b) presents 2$\times$ (top) and 4$\times$ (bottom) super-resolution results on Seek Mosaic data. Our models produce cleaner outputs with preserved structure and thermal continuity, effectively suppressing noise and artifacts across both tasks.
• Figure 5: Quantitative comparison of hyperparameter settings for denoising performance.