Low-light Stereo Image Enhancement and De-noising in the Low-frequency Information Enhanced Image Space

TL;DR

This paper proposes a method to perform enhancement and de-noising simultaneously, and shows that this method obtains better detail recovery and noise removal compared with state-of-the-art methods.

Abstract

Unlike single image task, stereo image enhancement can use another view information, and its key stage is how to perform cross-view feature interaction to extract useful information from another view. However, complex noise in low-light image and its impact on subsequent feature encoding and interaction are ignored by the existing methods. In this paper, a method is proposed to perform enhancement and de-noising simultaneously. First, to reduce unwanted noise interference, a low-frequency information enhanced module (IEM) is proposed to suppress noise and produce a new image space. Additionally, a cross-channel and spatial context information mining module (CSM) is proposed to encode long-range spatial dependencies and to enhance inter-channel feature interaction. Relying on CSM, an encoder-decoder structure is constructed, incorporating cross-view and cross-scale feature interactions to perform enhancement in the new image space. Finally, the network is trained with the constraints of both spatial and frequency domain losses. Extensive experiments on both synthesized and real datasets show that our method obtains better detail recovery and noise removal compared with state-of-the-art methods. In addition, a real stereo image enhancement dataset is captured with stereo camera ZED2. The code and dataset are publicly available at: https://www.github.com/noportraits/LFENet.

Figures (8)

• Figure 1: The overall framework of our proposed stereo image enhancement method, which contains two weights-shared branches to process left and right views respectively. The method includes three main modules, i.e., IEM, CVMI and CSFI. IEM takes in low-light images and its low-frequency part to suppress noise. CVMI performs cross-view feature interaction and CSFI performs interactions of multi-scale features of single view.
• Figure 2: The detailed process of the low-frequency information enhanced module(IEM). The part highlighted by the dashed line is channel attention.
• Figure 3: The detailed structure of feature encoding and decoding. (a): The structure of cross-channel and spatial context information mining module(CSM); (b): The main branch of encoder-decoder module.
• Figure 4: The left part is the details of Cross-View Matching and Interaction Module(CVMI), where only one scale of feature interaction is shown. The right part is the details of Cross-Scale Feature Interaction Module(CSFI).
• Figure 5: Visualization of the enhanced image of each method on Holopix50 dataset. It can be seen that compared with other methods, our method restores the texture and color information better.