Thermal Chameleon: Task-Adaptive Tone-mapping for Radiometric Thermal-Infrared images

TL;DR

Given the same image, TCNet tone-maps different representations of TIR images tailored for each specific task, eliminating the heuristic image rescaling preprocessing and reliance on the extensive prior knowledge of the scene temperature or task-specific characteristics.

Abstract

Thermal Infrared (TIR) imaging provides robust perception for navigating in challenging outdoor environments but faces issues with poor texture and low image contrast due to its 14/16-bit format. Conventional methods utilize various tone-mapping methods to enhance contrast and photometric consistency of TIR images, however, the choice of tone-mapping is largely dependent on knowing the task and temperature dependent priors to work well. In this paper, we present Thermal Chameleon Network (TCNet), a task-adaptive tone-mapping approach for RAW 14-bit TIR images. Given the same image, TCNet tone-maps different representations of TIR images tailored for each specific task, eliminating the heuristic image rescaling preprocessing and reliance on the extensive prior knowledge of the scene temperature or task-specific characteristics. TCNet exhibits improved generalization performance across object detection and monocular depth estimation, with minimal computational overhead and modular integration to existing architectures for various tasks. Project Page: https://github.com/donkeymouse/ThermalChameleon

Figures (7)

• Figure 1: (a): Conventional methods use a single TIR representation for all tasks (b): TCNet is a learning-based tone-mapping network for TIR images. Given the same network, TCNet achieves different tone-mapping, tailored for each task.
• Figure 2: Overview of Thermal Chameleon Network. The network consists of two stages: multichannel thermal embedding (green) which formulates diverse representations from a 14-bit radiometric TIR images and an adaptive channel compression network (blue) that adaptively compresses these multichannel thermal embeddings into 3 channels which is then used to be trained on downstream tasks. All components are trained in an end-to-end manner.
• Figure 3: Adaptive channel compression. Given multichannel TIR embedding, the network assigns adaptive weightings for each thermal embedding optimized by the task-dependent loss. Using such weightings, we adaptively tone-map TIR images via weighted average summation of each thermal embedding and the respective weights.
• Figure 4: Visualization examples of commonly used baseline TIR tone-mapping methods. $D$ value of 3 is used for thermal embedding
• Figure 5: Average image histogram distribution of the tone-mapped images across different datasets. Different tasks tone-map the same TIR images differently.