Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CSIM: A Copula-based similarity index sensitive to local changes for Image quality assessment

Safouane El Ghazouali, Umberto Michelucci, Yassin El Hillali, Hichem Nouira

TL;DR

The novel CSIM metric uses Gaussian Copula from probability theory to transform an image into vectors of pixel distribution associated to local image patches, enabling a more nuanced comparison of image patches making it more sensitive to local changes compared to other metrics.

Abstract

Image similarity metrics play an important role in computer vision applications, as they are used in image processing, computer vision and machine learning. Furthermore, those metrics enable tasks such as image retrieval, object recognition and quality assessment, essential in fields like healthcare, astronomy and surveillance. Existing metrics, such as PSNR, MSE, SSIM, ISSM and FSIM, often face limitations in terms of either speed, complexity or sensitivity to small changes in images. To address these challenges, a novel image similarity metric, namely CSIM, that combines real-time while being sensitive to subtle image variations is investigated in this paper. The novel metric uses Gaussian Copula from probability theory to transform an image into vectors of pixel distribution associated to local image patches. These vectors contain, in addition to intensities and pixel positions, information on the dependencies between pixel values, capturing the structural relationships within the image. By leveraging the properties of Copulas, CSIM effectively models the joint distribution of pixel intensities, enabling a more nuanced comparison of image patches making it more sensitive to local changes compared to other metrics. Experimental results demonstrate that CSIM outperforms existing similarity metrics in various image distortion scenarios, including noise, compression artifacts and blur. The metric's ability to detect subtle differences makes it suitable for applications requiring high precision, such as medical imaging, where the detection of minor anomalies can be of a high importance. The results obtained in this work can be reproduced from this Github repository: https://github.com/safouaneelg/copulasimilarity.

CSIM: A Copula-based similarity index sensitive to local changes for Image quality assessment

TL;DR

The novel CSIM metric uses Gaussian Copula from probability theory to transform an image into vectors of pixel distribution associated to local image patches, enabling a more nuanced comparison of image patches making it more sensitive to local changes compared to other metrics.

Abstract

Image similarity metrics play an important role in computer vision applications, as they are used in image processing, computer vision and machine learning. Furthermore, those metrics enable tasks such as image retrieval, object recognition and quality assessment, essential in fields like healthcare, astronomy and surveillance. Existing metrics, such as PSNR, MSE, SSIM, ISSM and FSIM, often face limitations in terms of either speed, complexity or sensitivity to small changes in images. To address these challenges, a novel image similarity metric, namely CSIM, that combines real-time while being sensitive to subtle image variations is investigated in this paper. The novel metric uses Gaussian Copula from probability theory to transform an image into vectors of pixel distribution associated to local image patches. These vectors contain, in addition to intensities and pixel positions, information on the dependencies between pixel values, capturing the structural relationships within the image. By leveraging the properties of Copulas, CSIM effectively models the joint distribution of pixel intensities, enabling a more nuanced comparison of image patches making it more sensitive to local changes compared to other metrics. Experimental results demonstrate that CSIM outperforms existing similarity metrics in various image distortion scenarios, including noise, compression artifacts and blur. The metric's ability to detect subtle differences makes it suitable for applications requiring high precision, such as medical imaging, where the detection of minor anomalies can be of a high importance. The results obtained in this work can be reproduced from this Github repository: https://github.com/safouaneelg/copulasimilarity.
Paper Structure (17 sections, 12 equations, 11 figures, 1 algorithm)

This paper contains 17 sections, 12 equations, 11 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Similarity metrics: state-of-the-art
  3. Structural Similarity Index (SSIM)
  4. Feature Similarity Index (FSIM)
  5. Information theoretic-based Statistic Similarity (ISSM)
  6. Proposed method: Copula-based Similarity (CSIM)
  7. Patching and features sorting
  8. Image Copula computing
  9. Copula Vectors Generation
  10. Euclidean distance calculation
  11. Experiments
  12. Performance evaluation under blurring and Gaussian noise
  13. Video-based dynamic assessment
  14. CSIQ Dataset: performance evaluation
  15. Discussion
  16. ...and 2 more sections

Figures (11)

  • Figure 1: Flowchart of the copula-based similarity metric (CSIM) for image comparison distributed on four steps: (A) Patching: includes color channel computing, features extracting and sorting. (B) Copula patch: incorporates rank calculation, CDF and PPF. (C) Copula vectors: merge the copula patches into a vector for each image. (D) Similarity Map: Includes the calculated distances between the copula vectors and similarity scoring.
  • Figure 2: Visualization example of noise and blur application on an RGB image: (a) Original image, (b) addition of white Gaussian noise with a mean of 5 and standard deviation of 10, (c) blurred version of the noisy image with a standard deviation of 10 and kernel size of (5,5).
  • Figure 3: Visualization of feature extraction and ranking along with and copula vectors computation through CDF and PPF: (a) Histograms of the pixel intensity distributions for each color channel of the image. (b) Histograms of the ranks for each color channel. Statistical analysis of the copula-transformed pixel intensities for each color channel: (c) CDF of the red, green and blue channel copula. (d) PPF of the red, green and blue channel copula.
  • Figure 4: Comparison of image quality metrics under varying conditions: (a) blurring levels with sigma varying from 0 to 20 and (b) noise levels with a mean of 5 and standard deviations varying from 0 to 20.
  • Figure 5: 3D visualizations of image quality metrics as functions of blur sigma and noise mean: (a) CSIM, (b) SSIM, (c) FSIM and (d) ISSM.
  • ...and 6 more figures