Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing the image correction pipeline for pedestrian detection in the thermal-infrared domain

Christophe Karam, Jessy Matias, Xavier Breniere, Jocelyn Chanussot

TL;DR

The paper addresses optimizing infrared image correction pipelines for pedestrian detection in urban autonomous driving settings. It systematically evaluates shutter-based and shutterless NUC pipelines, with a range of denoising and tonemapping options, using Lynred's VRU data and additional public datasets; detectors are built with YOLOv4-Tiny and accelerated via ONNX/TensorRT. Findings show that raw infrared is insufficient and tone-mapping and destriping play key roles, while spatial denoising degrades detection and temporal denoising offers selective gains. The main practical contribution is a recommended lean shutterless pipeline using tonemapping alone to balance speed and accuracy, enabling real-time autonomous driving across varied environments. The work highlights the importance of optimizing the signal-processing chain for neural detectors rather than visual quality alone.

Abstract

Infrared imagery can help in low-visibility situations such as fog and low-light scenarios, but it is prone to thermal noise and requires further processing and correction. This work studies the effect of different infrared processing pipelines on the performance of a pedestrian detection in an urban environment, similar to autonomous driving scenarios. Detection on infrared images is shown to outperform that on visible images, but the infrared correction pipeline is crucial since the models cannot extract information from raw infrared images. Two thermal correction pipelines are studied, the shutter and the shutterless pipes. Experiments show that some correction algorithms like spatial denoising are detrimental to performance even if they increase visual quality for a human observer. Other algorithms like destriping and, to a lesser extent, temporal denoising, increase computational time, but have some role to play in increasing detection accuracy. As it stands, the optimal trade-off for speed and accuracy is simply to use the shutterless pipe with a tonemapping algorithm only, for autonomous driving applications within varied environments.

Optimizing the image correction pipeline for pedestrian detection in the thermal-infrared domain

TL;DR

The paper addresses optimizing infrared image correction pipelines for pedestrian detection in urban autonomous driving settings. It systematically evaluates shutter-based and shutterless NUC pipelines, with a range of denoising and tonemapping options, using Lynred's VRU data and additional public datasets; detectors are built with YOLOv4-Tiny and accelerated via ONNX/TensorRT. Findings show that raw infrared is insufficient and tone-mapping and destriping play key roles, while spatial denoising degrades detection and temporal denoising offers selective gains. The main practical contribution is a recommended lean shutterless pipeline using tonemapping alone to balance speed and accuracy, enabling real-time autonomous driving across varied environments. The work highlights the importance of optimizing the signal-processing chain for neural detectors rather than visual quality alone.

Abstract

Infrared imagery can help in low-visibility situations such as fog and low-light scenarios, but it is prone to thermal noise and requires further processing and correction. This work studies the effect of different infrared processing pipelines on the performance of a pedestrian detection in an urban environment, similar to autonomous driving scenarios. Detection on infrared images is shown to outperform that on visible images, but the infrared correction pipeline is crucial since the models cannot extract information from raw infrared images. Two thermal correction pipelines are studied, the shutter and the shutterless pipes. Experiments show that some correction algorithms like spatial denoising are detrimental to performance even if they increase visual quality for a human observer. Other algorithms like destriping and, to a lesser extent, temporal denoising, increase computational time, but have some role to play in increasing detection accuracy. As it stands, the optimal trade-off for speed and accuracy is simply to use the shutterless pipe with a tonemapping algorithm only, for autonomous driving applications within varied environments.
Paper Structure (22 sections, 7 figures, 4 tables)

This paper contains 22 sections, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Infrared Correction Pipelines
  4. Infrared Imaging Applications
  5. Methods
  6. Datasets
  7. Infrared Image Correction
  8. Object Detection Models
  9. Results
  10. Experimental Setup
  11. Choice of Test Datasets
  12. The role of infrared and correction pipelines
  13. The shutter correction pipeline
  14. Calibration Temperatures
  15. Tonemapping
  16. ...and 7 more sections

Figures (7)

  • Figure 1: General workflow for optimizing the infrared processing pipeline based on pedestrian detection performance.
  • Figure 2: Image with annotated bounding boxes from Lynred VRU dataset across different domains: raw infrared, corrected infrared, and visible RGB.
  • Figure 3: Image from KAIST with our model's predictions, marked as true or false positives based on the actual but erroneous "groundtruth" annotations.
  • Figure 4: Test performances for a varying ambient calibration temperature for a shutter pipe with piecewise tonemapping
  • Figure 5: Test performances for a varying tonemapping algorithms with shutter pipe at 25℃ ambient temperature.
  • ...and 2 more figures