Image color consistency in datasets: the Smooth-TPS3D method

TL;DR

This work proposes an improved 3D Thin-Plate Splines (TPS3D) color correction method to be used, in conjunction with color charts or other machine-readable patterns, to achieve image consistency by post-processing.

Abstract

Image color consistency is the key problem in digital imaging consistency when creating datasets. Here, we propose an improved 3D Thin-Plate Splines (TPS3D) color correction method to be used, in conjunction with color charts (i.e. Macbeth ColorChecker) or other machine-readable patterns, to achieve image consistency by post-processing. Also, we benchmark our method against its former implementation and the alternative methods reported to date with an augmented dataset based on the Gehler's ColorChecker dataset. Benchmark includes how corrected images resemble the ground-truth images and how fast these implementations are. Results demonstrate that the TPS3D is the best candidate to achieve image consistency. Furthermore, our Smooth-TPS3D method shows equivalent results compared to the original method and reduced the 11-15% of ill-conditioned scenarios which the previous method failed to less than 1%. Moreover, we demonstrate that the Smooth-TPS method is 20% faster than the original method. Finally, we discuss how different methods offer different compromises between quality, correction accuracy and computational load.

Figures (6)

• Figure 1: State-of-the-art color correction charts from X-Rite, Pantone. (a) The X-Rite ColorChecker Passport Photo 2® kit. (b) The Pantone Color Match Card®.
• Figure 2: Different examples of color augmentation using imgaug in Python. The first image on the left is the developed original image from the Gehlre’s dataset. The other images are augmentations of these image with variations in color, contrast and saturation.
• Figure 3: A count of the failed corrections for each correction method is shown. Failed corrections are selected if their $\overline{\Delta_{RGB}}_{,within}$ computation is greater than the NONE correction. After this, the count is divided in ill-conditioned results or not. Ill-condition is assessed using the $\mathbf{\Delta_{RGB}}_{,pairwise}$ comparison to a minimum distance of $\Delta_{RGB} = \sqrt{3}$.
• Figure 4: The $\overline{\Delta_{RGB}}_{,within}$ for each image in the dataset is shown as a distribution against the color correction techniques. The distribution information is represented as follows: ($\triangle$) the mean, ($-$) the median at $\mathrm{Q2}$, (box) from $\mathrm{Q1}$ to $\mathrm{Q3}$, (whiskers) $\mathrm{1.5 \cdot (Q3 - Q1)}$ into $\mathrm{Q0}$ and $\mathrm{Q4}$ and ($\circ$) the outliers. PERF correction is not zero and shows the quantization effect. NONE is a reference of not applying any correction at all. The rest of the corrections are grouped in: AFF, VAN, CHE, FIN and TPS corrections.
• Figure 5: The $\overline{\Delta_{RGB}}_{,inter}$ for each image in the dataset is shown a distribution against the color correction techniques. The distribution information is represented as follows: ($\triangle$) the mean, ($-$) the median at $\mathrm{Q2}$, (box) from $\mathrm{Q1}$ to $\mathrm{Q3}$, (whiskers) $\mathrm{1.5 \cdot (Q3 - Q1)}$ into $\mathrm{Q0}$ and $\mathrm{Q4}$ and ($\circ$) the outliers. PERF correction is not zero and shows the quantization effect. NONE is a reference of not applying any correction at all. The rest of the corrections are grouped in: AFF, VAN, CHE, FIN and TPS corrections.