Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MIPI 2024 Challenge on Demosaic for HybridEVS Camera: Methods and Results

Yaqi Wu, Zhihao Fan, Xiaofeng Chu, Jimmy S. Ren, Xiaoming Li, Zongsheng Yue, Chongyi Li, Shangcheng Zhou, Ruicheng Feng, Yuekun Dai, Peiqing Yang, Chen Change Loy, Senyan Xu, Zhijing Sun, Jiaying Zhu, Yurui Zhu, Xueyang Fu, Zheng-Jun Zha, Jun Cao, Cheng Li, Shu Chen, Liang Ma, Shiyang Zhou, Haijin Zeng, Kai Feng, Yongyong Chen, Jingyong Su, Xianyu Guan, Hongyuan Yu, Cheng Wan, Jiamin Lin, Binnan Han, Yajun Zou, Zhuoyuan Wu, Yuan Huang, Yongsheng Yu, Daoan Zhang, Jizhe Li, Xuanwu Yin, Kunlong Zuo, Yunfan Lu, Yijie Xu, Wenzong Ma, Weiyu Guo, Hui Xiong, Wei Yu, Bingchun Luo, Sabari Nathan, Priya Kansal

TL;DR

The paper documents the MIPI 2024 Demosaic for HybridEVS Camera challenge, focusing on reconstructing high-quality RGB images from HybridEVS input that includes event and defect pixels. It introduces a dataset (800 training pairs at 2K, 50 validation, 50 testing scenes) and standard evaluation (PSNR/SSIM) while outlining a three-phase challenge (Development, Validation, Testing). Seven teams propose diverse architectures, ranging from coarse-to-fine transformers to two-stage inpainting/demosaicing and event-focused processing, achieving state-of-the-art PSNR around 44.8 dB and SSIM ~0.985 on real test data. The results demonstrate strong progress in HybridEVS demosaicing, contribute novel architectural ideas, and establish benchmarks to spur further research and applications in mobile computational photography.

Abstract

The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems. However, the scarcity of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). Building on the achievements of the previous MIPI Workshops held at ECCV 2022 and CVPR 2023, we introduce our third MIPI challenge including three tracks focusing on novel image sensors and imaging algorithms. In this paper, we summarize and review the Nighttime Flare Removal track on MIPI 2024. In total, 170 participants were successfully registered, and 14 teams submitted results in the final testing phase. The developed solutions in this challenge achieved state-of-the-art performance on Nighttime Flare Removal. More details of this challenge and the link to the dataset can be found at https://mipi-challenge.org/MIPI2024/.

MIPI 2024 Challenge on Demosaic for HybridEVS Camera: Methods and Results

TL;DR

The paper documents the MIPI 2024 Demosaic for HybridEVS Camera challenge, focusing on reconstructing high-quality RGB images from HybridEVS input that includes event and defect pixels. It introduces a dataset (800 training pairs at 2K, 50 validation, 50 testing scenes) and standard evaluation (PSNR/SSIM) while outlining a three-phase challenge (Development, Validation, Testing). Seven teams propose diverse architectures, ranging from coarse-to-fine transformers to two-stage inpainting/demosaicing and event-focused processing, achieving state-of-the-art PSNR around 44.8 dB and SSIM ~0.985 on real test data. The results demonstrate strong progress in HybridEVS demosaicing, contribute novel architectural ideas, and establish benchmarks to spur further research and applications in mobile computational photography.

Abstract

The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems. However, the scarcity of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). Building on the achievements of the previous MIPI Workshops held at ECCV 2022 and CVPR 2023, we introduce our third MIPI challenge including three tracks focusing on novel image sensors and imaging algorithms. In this paper, we summarize and review the Nighttime Flare Removal track on MIPI 2024. In total, 170 participants were successfully registered, and 14 teams submitted results in the final testing phase. The developed solutions in this challenge achieved state-of-the-art performance on Nighttime Flare Removal. More details of this challenge and the link to the dataset can be found at https://mipi-challenge.org/MIPI2024/.
Paper Structure (17 sections, 3 equations, 11 figures, 1 table)

This paper contains 17 sections, 3 equations, 11 figures, 1 table.

Table of Contents

  1. Introduction
  2. MIPI 2024 Demosaic for Hybridevs Camera
  3. Problem Definition
  4. Datasets
  5. Evaluation
  6. Challenge Phase
  7. Challenge Results
  8. Methods
  9. USTC604
  10. lolers
  11. Lumos Demosaicker
  12. High-speed Machines
  13. Yunfan
  14. HIT-CVLAB
  15. CougerAI
  16. ...and 2 more sections

Figures (11)

  • Figure 1: (a) Quad Bayer pattern, (b) HybridEVS pattern.
  • Figure 2: The Demosaic for the HybridEVS Camera aims to reconstruct HybridEVS data into a high-quality RGB result with the same resolution. This process involves passing the data through a demosaic module, which corrects defects and event pixels and reconstructs a three-channel RGB image of matching resolution.
  • Figure 3: Some sample images from the proposed training dataset. The scenes include natural landscapes, architectural views, and other such scenes.
  • Figure 4: Some sample images from the testing dataset. It includes various scenarios like indoor scenes and outdoor scenes
  • Figure 5: The architecture of DemosaicFormer proposed by team USTC604 to demosaic the raw data captured by HybridEVS cameras.
  • ...and 6 more figures