Review of Measures Used for Evaluating Color Difference Models

TL;DR

This work addresses the non-Euclidean nature of color perception by comparing a set of color-difference measures through a continuous, ellipsoid-based framework that integrates over all directions to remove sampling bias. It develops both continuous and transformation-based approaches, revealing that V_AB, γ-1, CV, and STRESS are equivalent for small deviations, while diverging for larger ones, with STRESS remaining coordinate-independent. The authors introduce an eigenvalue-based distance, $d_{ev}$, and show its close relationship to Pant's geometric measure $\bar{R}$, both offering simple, affine-invariant characterizations of perceptual difference. The results guide the selection and computation of color-difference metrics, highlighting the practical advantages of eigenvalue-based and Pant-inspired measures for robust, coordinate-invariant assessment in color appearance modeling and CAMs.

Abstract

We made a detailed review of the difference measures which have been used to judge the differences between experimentally determined color differences and theoretically defined ones, so-called line elements, for the human visual system. To eliminate the statistical errors due to variable and usually arbitrary sampling of the directions in a color point, we integrate the measures over a complete ellipsoid/ellipse. It turns out that in the limit for small deviations from circularity all proposed measures ($V_{AB}$, $γ-1$, $CV$ and $\mathrm{STRESS}$) are equivalent. For greater deviations the measures become distinct with $γ-1$ the most sensitive and $\mathrm{STRESS}$ the least. Ideally a difference measure should be coordinate independent and then it is advantageous to apply an affine transformation to both sets, e.g. turning the theoretical one into the unit ball. Although MacAdam already used this method but sampled the transformed ellipse, we integrate over the ellipsoid/ellipse. Comparing the results with the base measures we show that only $\mathrm{STRESS}$ is coordinate independent. Judging whether a single ellipsoid/ellipse resembles a unit ball can easily be done by comparing the eigenvalues with one and we show that our previously proposed error measure $d_{ev}$ (Candry e.a. Optics Express, 30, 36307, 2022) is the eigenvalue version of $γ-1$. We show why the short lived correlation coefficient $r$ was justly abandoned, being very coordinate dependent, but that Pant's recent geometric measure $1-R$ on the other hand is coordinate independent. All measures are routinely made scale invariant by the introduction of a scaling parameter, to be optimized. Lastly we show that from all measures the $γ-1$ ones are the only ones permitting the simple derivation of the globally optimized difference measure from the locally defined ones.

Figures (7)

• Figure 1: A generic (experimental) chromaticity ellipse at threshold in an arbitrary coordinate system $\left(x_{1},x_{2}\right)$. O is an arbitrary point of the color manifold and $P$ defines an arbitrary direction OP, defined by $\theta$. The orientation of the ellipse is defined by the angle $\Delta\theta$ of the long axis with the $x_{1}$-axis. The size of the ellipse is defined by the eigenvalues $\gamma_{1}<\gamma_{2}$.
• Figure 2: The continuous difference measures $\bar{\gamma}$ (solid), $V_{AB}$ (dash) , $\mathrm{STRESS}$ (dot) and $CV$ (dashdot) as a function of $\delta_{\lambda}$ for (i) $\delta_{\gamma}=\delta_{\lambda}$ and $\Delta\theta=\pi/2$ (blue curves) and for (ii) $\delta_{\gamma}=0$ and $\Delta\theta=0$ (red curves). In the 2nd case $\mathrm{STRESS}$ and $CV$ are identical and $V_{AB}$ almost identical. For the first case the 3 measures $V_{AB}$, $\mathrm{STRESS}$ and $CV$ are also very similar at least up to $\delta=0.9$, corresponding with an aspect ratio of $1/19$.
• Figure 3: Discrete difference measures $\bar{\gamma}$ (solid), $V_{AB}$ (dash) , $\mathrm{STRESS}$ (dot) and $CV$ (dashdot) for 2 sampled directions $\frac{\pi}{3}$ apart and for 2 rather elongated ellipses ($\delta_{\lambda}=\delta_{\gamma}=0.9$ , $\Delta\theta=\pi/4$). The red curve shows the average measure $PF/3$.
• Figure 4: The continuous correlation coefficient as a function of $\delta_{\lambda}=\delta_{\gamma}$ for $\Delta\theta=0,\frac{\pi}{8},\frac{\pi}{4},\frac{3\pi}{8}$ and $\frac{\pi}{2}$. To be useful as a distance measure one should consider $\bar{r}=1-r$ but the hypersensitivity for the orientation of the ellipses when they become almost circular explains why this measure is not fit for comparing color differences.
• Figure 5: The difference measures $\bar{\gamma}_{\mu}$ (solid), $V_{AB,\mu}$ (dash) , $\mathrm{STRESS}$ (dot) ${CV}_{\mu}$ (dashdot) $\bar{R}$ (dashed green line) and $d_{ev}$ (heavy red line) as a function of $\delta_{\mu}$ for ellipses. $V_{AB,\mu}$ and $\mathrm{STRESS}$ are almost coincident. The straight line is the mutual asymptote of the first 4 measures ($\frac{\delta_{\mu}}{2\sqrt{2}}$) for $\delta_{\mu}\rightarrow0$. The slopes of $d_{ev}$ ($\frac{\delta_{\mu}}{2}$) and of $\bar{R}$ ($\frac{2}{\pi}\delta_{\mu}$) are larger by factors $\sqrt{2}\approx1.4$ and $\frac{4\sqrt{2}}{\pi}\approx1.8$. Note that $d_{ev}$, $\mathrm{STRESS}$ and $\bar{R}$ are coordinate independent.