Troublemaker Learning for Low-Light Image Enhancement

TL;DR

The Trouble-Maker Learning (TML) strategy, which leverages images with normal light as training inputs, and the UGDC model, trained with TML and via data fusion, can achieve performance competitive with state-of-the-art approaches on public datasets.

Abstract

Low-light image enhancement (LLIE) restores the color and brightness of underexposed images. Supervised methods suffer from high costs in collecting low/normal-light image pairs. Unsupervised methods invest substantial effort in crafting complex loss functions. We address these two challenges through the proposed TroubleMaker Learning (TML) strategy, which employs normal-light images as inputs for training. TML is simple: we first dim the input and then increase its brightness. TML is based on two core components. First, the troublemaker model (TM) constructs pseudo low-light images from normal images to relieve the cost of pairwise data. Second, the predicting model (PM) enhances the brightness of pseudo low-light images. Additionally, we incorporate an enhancing model (EM) to further improve the visual performance of PM outputs. Moreover, in LLIE tasks, characterizing global element correlations is important because more information on the same object can be captured. CNN cannot achieve this well, and self-attention has high time complexity. Accordingly, we propose Global Dynamic Convolution (GDC) with O(n) time complexity, which essentially imitates the partial calculation process of self-attention to formulate elementwise correlations. Based on the GDC module, we build the UGDC model. Extensive quantitative and qualitative experiments demonstrate that UGDC trained with TML can achieve competitive performance against state-of-the-art approaches on public datasets. The code is available at https://github.com/Rainbowman0/TML_LLIE.

Figures (9)

• Figure 1: Three categories of training paradigms, SL, USL, and TML, denote supervised learning, unsupervised learning, and troublemaker learning, respectively. Supervised methods rely on paired data, while complex loss functions limit unsupervised methods. TML weakens the paired data restriction and exploits a simple loss function.
• Figure 2: Overview of TML. The left side shows the TML training and testing process. In the training phase, TM only serves as a pseudo label generator. Step 1 requires a small quantity of paired data for training TM. Step 2 freezes TM and trains only PM and EM. The testing phase involves PM and EM. The UGDC (TM, PM and EM structure) and GDC details are on the right. GDC imitates the partial calculation process of self-attention and corresponds with \ref{['eq04_2', 'eq05_2', 'eq06_2', 'eq07_2']}.
• Figure 3: Pseudo low-light images. The first and third columns are normal-light and real low-light images obtained via adjusting ISO. The second column is the pseudo low-light images predicted by TM.
• Figure 4: The matrix multiplication between $Q$ and $K$ in self-attention (top) can be replaced by a convolution operation (bottom). We use red and green to mark the equivalent elements in $Q$ and $Q'$, $K$ and $K'$, $A$ and $A'$.
• Figure 5: Qualitative comparisons with state-of-the-art methods. TML makes reasonable trade-offs in brightness, color, and image quality.