SLLEN: Semantic-aware Low-light Image Enhancement Network

TL;DR

Unlike currently available approaches, the proposed SLLEN is able to fully lever the semantic information, e.g., IEF, HSF, and SS dataset, to assist LLE, thereby leading to a more promising enhancement performance.

Abstract

How to effectively explore semantic feature is vital for low-light image enhancement (LLE). Existing methods usually utilize the semantic feature that is only drawn from the output produced by high-level semantic segmentation (SS) network. However, if the output is not accurately estimated, it would affect the high-level semantic feature (HSF) extraction, which accordingly interferes with LLE. To this end, we develop a simple and effective semantic-aware LLE network (SSLEN) composed of a LLE main-network (LLEmN) and a SS auxiliary-network (SSaN). In SLLEN, LLEmN integrates the random intermediate embedding feature (IEF), i.e., the information extracted from the intermediate layer of SSaN, together with the HSF into a unified framework for better LLE. SSaN is designed to act as a SS role to provide HSF and IEF. Moreover, thanks to a shared encoder between LLEmN and SSaN, we further propose an alternating training mechanism to facilitate the collaboration between them. Unlike currently available approaches, the proposed SLLEN is able to fully lever the semantic information, e.g., IEF, HSF, and SS dataset, to assist LLE, thereby leading to a more promising enhancement performance. Comparisons between the proposed SLLEN and other state-of-the-art techniques demonstrate the superiority of SLLEN with respect to LLE quality over all the comparable alternatives.

Figures (12)

• Figure 1: The comparisons between the proposed SLLEN and three most representative state-of-the-art techniques, including LIME (model-based) guo2016lime, DCE++ (learning-based) 9369102, and ISSR (semantic-guided learning-based) fan2020integrating. (a)$\sim$(e): Visual comparison on two given examples; As shown, the results of the proposed SLLEN are superior in terms of contrast, exposure, and color, while the results of others are either under-enhanced or over-enhanced. (f): The average scores of different methods in terms of PSNR, SSIM, LOE, CEIQ, and mIoU on several commonly-used datasets remarked in Subsections \ref{['section3-2']} and \ref{['section3-3']}; it can be easily noticed that our SLLEN remarkably outperforms these competitors.
• Figure 2: The system architecture for applying the proposed SLLEN to ITS. (a) is an intersection deployed with traffic cameras. (b) is the low-light image enhancement procedure of our SLLEN. (c) is the applications of enhanced images in ITS
• Figure 3: The architecture of the proposed SLLEN. It consists of LLE main-network (LLEmN) and SS auxiliary-network (SSaN), where LLEmN contains Feature Extraction Module (highlight in yellow), Feature Enhancement Module (highlight in green), Feature Fusion Module (highlight in purple).
• Figure 4: (a): Input low-light image and the corresponding inaccurate SS map produced by SSaN. (b): Intermediate embedding features extracted from $5^{th}$ layer of SSaN.
• Figure 5: (a): The framework of High-level Semantic Based Attention Block (HSBAB). (b): The framework of Random Semantic Aware Enhancement Block (RSAEB). (c): The framework of Feature Fusion Block (FFB).