Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maximum entropy and quantized metric models for absolute category ratings

Dietmar Saupe, Krzysztof Rusek, David Hägele, Daniel Weiskopf, Lucjan Janowski

TL;DR

This study investigates families of multinomial probability distributions parameterized by mean and variance that are used to fit the empirical rating distributions and introduces a novel discrete maximum entropy distribution for a given mean and variance.

Abstract

The datasets of most image quality assessment studies contain ratings on a categorical scale with five levels, from bad (1) to excellent (5). For each stimulus, the number of ratings from 1 to 5 is summarized and given in the form of the mean opinion score. In this study, we investigate families of multinomial probability distributions parameterized by mean and variance that are used to fit the empirical rating distributions. To this end, we consider quantized metric models based on continuous distributions that model perceived stimulus quality on a latent scale. The probabilities for the rating categories are determined by quantizing the corresponding random variables using threshold values. Furthermore, we introduce a novel discrete maximum entropy distribution for a given mean and variance. We compare the performance of these models and the state of the art given by the generalized score distribution for two large data sets, KonIQ-10k and VQEG HDTV. Given an input distribution of ratings, our fitted two-parameter models predict unseen ratings better than the empirical distribution. In contrast to empirical ACR distributions and their discrete models, our continuous models can provide fine-grained estimates of quantiles of quality of experience that are relevant to service providers to satisfy a target fraction of the user population.

Maximum entropy and quantized metric models for absolute category ratings

TL;DR

This study investigates families of multinomial probability distributions parameterized by mean and variance that are used to fit the empirical rating distributions and introduces a novel discrete maximum entropy distribution for a given mean and variance.

Abstract

The datasets of most image quality assessment studies contain ratings on a categorical scale with five levels, from bad (1) to excellent (5). For each stimulus, the number of ratings from 1 to 5 is summarized and given in the form of the mean opinion score. In this study, we investigate families of multinomial probability distributions parameterized by mean and variance that are used to fit the empirical rating distributions. To this end, we consider quantized metric models based on continuous distributions that model perceived stimulus quality on a latent scale. The probabilities for the rating categories are determined by quantizing the corresponding random variables using threshold values. Furthermore, we introduce a novel discrete maximum entropy distribution for a given mean and variance. We compare the performance of these models and the state of the art given by the generalized score distribution for two large data sets, KonIQ-10k and VQEG HDTV. Given an input distribution of ratings, our fitted two-parameter models predict unseen ratings better than the empirical distribution. In contrast to empirical ACR distributions and their discrete models, our continuous models can provide fine-grained estimates of quantiles of quality of experience that are relevant to service providers to satisfy a target fraction of the user population.
Paper Structure (11 sections, 3 equations, 1 figure, 3 tables)

This paper contains 11 sections, 3 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Models
  3. Quantized metric models
  4. The maximum entropy model
  5. Generalized score distribution model (GSD)
  6. Empirical frequency distribution
  7. Methods
  8. Maximum likelihood estimation and goodness of fit
  9. Prediction accuracy for unseen data
  10. Results and discussion
  11. Conclusion

Figures (1)

  • Figure 1: The goodness of the model fit is shown by the G-test CDFs (left and center). Curves above the dashed line for the asymptotic chi-squared distribution indicate a statistically valid fit. The right figure shows that the logit-logistic and the GSD models predict unseen data better than the empirical model from samples of 10 to 40 ACR ratings of KonIQ-10k. The prediction errors are given as $L^{\infty}$-distances of the corresponding distributions (see Table \ref{['table_pred_logistic']} for more details).