Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Real-Time Framework for Domain-Adaptive Underwater Object Detection with Image Enhancement

Junjie Wen, Jinqiang Cui, Benyun Zhao, Bingxin Han, Xuchen Liu, Zhi Gao, Ben M. Chen

TL;DR

EnYOLO, an integrated real-time framework designed for simultaneous UIE and UOD with domain-adaptation capability, is introduced, which achieves state-of-the-art performance in both UIE and UOD tasks, but also shows superior adaptability when applied to different underwater scenarios.

Abstract

In recent years, significant progress has been made in the field of underwater image enhancement (UIE). However, its practical utility for high-level vision tasks, such as underwater object detection (UOD) in Autonomous Underwater Vehicles (AUVs), remains relatively unexplored. It may be attributed to several factors: (1) Existing methods typically employ UIE as a pre-processing step, which inevitably introduces considerable computational overhead and latency. (2) The process of enhancing images prior to training object detectors may not necessarily yield performance improvements. (3) The complex underwater environments can induce significant domain shifts across different scenarios, seriously deteriorating the UOD performance. To address these challenges, we introduce EnYOLO, an integrated real-time framework designed for simultaneous UIE and UOD with domain-adaptation capability. Specifically, both the UIE and UOD task heads share the same network backbone and utilize a lightweight design. Furthermore, to ensure balanced training for both tasks, we present a multi-stage training strategy aimed at consistently enhancing their performance. Additionally, we propose a novel domain-adaptation strategy to align feature embeddings originating from diverse underwater environments. Comprehensive experiments demonstrate that our framework not only achieves state-of-the-art (SOTA) performance in both UIE and UOD tasks, but also shows superior adaptability when applied to different underwater scenarios. Our efficiency analysis further highlights the substantial potential of our framework for onboard deployment.

A Real-Time Framework for Domain-Adaptive Underwater Object Detection with Image Enhancement

TL;DR

EnYOLO, an integrated real-time framework designed for simultaneous UIE and UOD with domain-adaptation capability, is introduced, which achieves state-of-the-art performance in both UIE and UOD tasks, but also shows superior adaptability when applied to different underwater scenarios.

Abstract

In recent years, significant progress has been made in the field of underwater image enhancement (UIE). However, its practical utility for high-level vision tasks, such as underwater object detection (UOD) in Autonomous Underwater Vehicles (AUVs), remains relatively unexplored. It may be attributed to several factors: (1) Existing methods typically employ UIE as a pre-processing step, which inevitably introduces considerable computational overhead and latency. (2) The process of enhancing images prior to training object detectors may not necessarily yield performance improvements. (3) The complex underwater environments can induce significant domain shifts across different scenarios, seriously deteriorating the UOD performance. To address these challenges, we introduce EnYOLO, an integrated real-time framework designed for simultaneous UIE and UOD with domain-adaptation capability. Specifically, both the UIE and UOD task heads share the same network backbone and utilize a lightweight design. Furthermore, to ensure balanced training for both tasks, we present a multi-stage training strategy aimed at consistently enhancing their performance. Additionally, we propose a novel domain-adaptation strategy to align feature embeddings originating from diverse underwater environments. Comprehensive experiments demonstrate that our framework not only achieves state-of-the-art (SOTA) performance in both UIE and UOD tasks, but also shows superior adaptability when applied to different underwater scenarios. Our efficiency analysis further highlights the substantial potential of our framework for onboard deployment.
Paper Structure (22 sections, 5 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 22 sections, 5 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Underwater Image Enhancement
  4. Underwater Object Detection
  5. Underwater Domain Adaptation
  6. Proposed Method
  7. Problem Definition
  8. Network Overview
  9. Multi-Stage Training Strategy
  10. Loss Design
  11. Burn-In Stage
  12. Mutual-Learning Stage
  13. Domain-Adaptation Stage
  14. EXPERIMENTS
  15. Implementation Details
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Visualization of detection results in greenish, bluish, and turbid underwater environments. Our proposed EnYOLO conducts simultaneous UIE and UOD effectively. Yellow dotted rectangles indicate missed detections, while red dotted ones represent incorrect detections. Zoom in for a better view.
  • Figure 2: Overview of our proposed EnYOLO framework. (a) The schematic illustration of training process. (b) The inference process.
  • Figure 3: Network Architecture for UIE task.
  • Figure 4: Visual comparison between various UIE methods. The first two rows contain sample images from the DUO test set liu2021dataset, and the last two rows contain sample images from the UIEB dataset li2019underwater. Zoom in for a better view.
  • Figure 5: Visualization of tSNE feature embeddings from various underwater images. Green, blue, and orange dots denote features of images from greenish, bluish, and turbid underwater environments, respectively.