Illumination Histogram Consistency Metric for Quantitative Assessment of Video Sequences

TL;DR

Illumination inconsistency in video sequences lacks a standard quantitative metric. The authors introduce Illumination Histogram Consistency (IHC), which uses Per-frame illumination maps estimated by the Retinex model and per-frame illumination histograms to quantify inter-frame illumination variation. They define $IHD = \frac{\sum_{i=1}^K\sum_{j=0}^{255}|G_i(j)-M(j)|}{K\,S}$ and $IHC = 2 - IHD$, where $G_i(j)$ are per-frame histograms and $M(j)$ is the mean histogram; they validate on Blender-generated sequences, showing $IHD$ grows with larger frame intervals and $IHC$ correspondingly decreases, matching perceptual guidance. The method is simple, fast, and applicable to dispersed image sets beyond consecutive video frames, with code released for reproducibility.

Abstract

The advances in deep generative models have greatly accelerate the process of video procession such as video enhancement and synthesis. Learning spatio-temporal video models requires to capture the temporal dynamics of a scene, in addition to the visual appearance of individual frames. Illumination consistency, which reflects the variations of illumination in the dynamic video sequences, play a vital role in video processing. Unfortunately, to date, no well-accepted quantitative metric has been proposed for video illumination consistency evaluation. In this paper, we propose a illumination histogram consistency (IHC) metric to quantitatively and automatically evaluate the illumination consistency of the video sequences. IHC measures the illumination variation of any video sequence based on the illumination histogram discrepancies across all the frames in the video sequence. Specifically, given a video sequence, we first estimate the illumination map of each individual frame using the Retinex model; Then, using the illumination maps, the mean illumination histogram of the video sequence is computed by the mean operation across all the frames; Next, we compute the illumination histogram discrepancy between each individual frame and the mean illumination histogram and sum up all the illumination histogram discrepancies to represent the illumination variations of the video sequence. Finally, we obtain the IHC score from the illumination histogram discrepancies via normalization and subtraction operations. Experiments are conducted to illustrate the performance of the proposed IHC metric and its capability to measure the illumination variations in video sequences. The source code is available on \url{https://github.com/LongChenCV/IHC-Metric}.

Figures (4)

• Figure 1: The proposed illumination histogram consistency (IHC) metric (a). For the video sequence, IHC first estimates the illumination map using the Retinex Model (b). Then, it exploits the consistency of illumination histogram (c) to measure the illumination consistency of the video sequence.
• Figure 2: The IHC scores of image sets generated by different frame interval settings. 'FID' indicates the frame ID in the video sequence with linearly increasing illumination density. 'Interval' indicates the interval between two adjacent frames.
• Figure 3: The raw images, the reflectance maps, the illumination maps and the illumination histograms generated under the setting of $Interval=6$.
• Figure 4: Larger frame intervals bring higher IHD score and lower IHC scores since more illumination variations are introduced by larger intervals.