Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IndraEye: Infrared Electro-Optical UAV-based Perception Dataset for Robust Downstream Tasks

Manjunath D, Prajwal Gurunath, Sumanth Udupa, Aditya Gandhamal, Shrikar Madhu, Aniruddh Sikdar, Suresh Sundaram

TL;DR

The IndraEye dataset is introduced, a multi-sensor (EO-IR) dataset designed for various tasks, which includes 5,612 images with 145,666 instances, encompassing multiple viewing angles, altitudes, seven backgrounds, and different times of the day across the Indian subcontinent.

Abstract

Deep neural networks (DNNs) have shown exceptional performance when trained on well-illuminated images captured by Electro-Optical (EO) cameras, which provide rich texture details. However, in critical applications like aerial perception, it is essential for DNNs to maintain consistent reliability across all conditions, including low-light scenarios where EO cameras often struggle to capture sufficient detail. Additionally, UAV-based aerial object detection faces significant challenges due to scale variability from varying altitudes and slant angles, adding another layer of complexity. Existing methods typically address only illumination changes or style variations as domain shifts, but in aerial perception, correlation shifts also impact DNN performance. In this paper, we introduce the IndraEye dataset, a multi-sensor (EO-IR) dataset designed for various tasks. It includes 5,612 images with 145,666 instances, encompassing multiple viewing angles, altitudes, seven backgrounds, and different times of the day across the Indian subcontinent. The dataset opens up several research opportunities, such as multimodal learning, domain adaptation for object detection and segmentation, and exploration of sensor-specific strengths and weaknesses. IndraEye aims to advance the field by supporting the development of more robust and accurate aerial perception systems, particularly in challenging conditions. IndraEye dataset is benchmarked with object detection and semantic segmentation tasks. Dataset and source codes are available at https://bit.ly/indraeye.

IndraEye: Infrared Electro-Optical UAV-based Perception Dataset for Robust Downstream Tasks

TL;DR

The IndraEye dataset is introduced, a multi-sensor (EO-IR) dataset designed for various tasks, which includes 5,612 images with 145,666 instances, encompassing multiple viewing angles, altitudes, seven backgrounds, and different times of the day across the Indian subcontinent.

Abstract

Deep neural networks (DNNs) have shown exceptional performance when trained on well-illuminated images captured by Electro-Optical (EO) cameras, which provide rich texture details. However, in critical applications like aerial perception, it is essential for DNNs to maintain consistent reliability across all conditions, including low-light scenarios where EO cameras often struggle to capture sufficient detail. Additionally, UAV-based aerial object detection faces significant challenges due to scale variability from varying altitudes and slant angles, adding another layer of complexity. Existing methods typically address only illumination changes or style variations as domain shifts, but in aerial perception, correlation shifts also impact DNN performance. In this paper, we introduce the IndraEye dataset, a multi-sensor (EO-IR) dataset designed for various tasks. It includes 5,612 images with 145,666 instances, encompassing multiple viewing angles, altitudes, seven backgrounds, and different times of the day across the Indian subcontinent. The dataset opens up several research opportunities, such as multimodal learning, domain adaptation for object detection and segmentation, and exploration of sensor-specific strengths and weaknesses. IndraEye aims to advance the field by supporting the development of more robust and accurate aerial perception systems, particularly in challenging conditions. IndraEye dataset is benchmarked with object detection and semantic segmentation tasks. Dataset and source codes are available at https://bit.ly/indraeye.

Paper Structure

This paper contains 13 sections, 2 figures, 8 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Aerial IndraEye Dataset
  4. Acquisition process
  5. Camera calibration
  6. Statistics of the dataset
  7. Class labels
  8. Ethics and policy
  9. EXPERIMENTAL ANALYSIS
  10. Object Detection
  11. Semantic Segmentation
  12. Domain adaptation and generalization
  13. CONCLUSION

Figures (2)

  • Figure 1: Drone captured EO and IR image pairs at high altitude and look angles (T1 and T2) with varying slant perspectives, enhancing multi-modal aerial perception of semi-urban traffic scene.
  • Figure 2: Snapshots from the IndraEye dataset showing different modalities EO, IR and complete semantic annotations for detection & segmentation tasks taken from different slant angles