HDRSDR-VQA: A Subjective Video Quality Dataset for HDR and SDR Comparative Evaluation

TL;DR

HDRSDR-VQA addresses the need for large-scale, content-level HDR vs SDR VQA by introducing 960 videos from 54 source sequences, rendered in HDR10 and SDR across nine quality levels, and evaluated through a 145-participant subjective study on six HDR-capable TVs. Using pairwise comparisons and ASAP active sampling, the study yields $JOD$ scores via pwcmp, enabling fine-grainedHDR/SDR comparisons under realistic viewing. The results show HDR provides perceptual advantages for content with rich textures and high brightness/color gamut, but these advantages diminish or reverse in low-dynamic or motion-heavy scenes, highlighting the content-dependence of HDR benefits. The dataset offers a valuable resource for benchmarking objective metrics and guiding perceptual optimization in HDR encoding and adaptive streaming systems.

Abstract

We introduce HDRSDR-VQA, a large-scale video quality assessment dataset designed to facilitate comparative analysis between High Dynamic Range (HDR) and Standard Dynamic Range (SDR) content under realistic viewing conditions. The dataset comprises 960 videos generated from 54 diverse source sequences, each presented in both HDR and SDR formats across nine distortion levels. To obtain reliable perceptual quality scores, we conducted a comprehensive subjective study involving 145 participants and six consumer-grade HDR-capable televisions. A total of over 22,000 pairwise comparisons were collected and scaled into Just-Objectionable-Difference (JOD) scores. Unlike prior datasets that focus on a single dynamic range format or use limited evaluation protocols, HDRSDR-VQA enables direct content-level comparison between HDR and SDR versions, supporting detailed investigations into when and why one format is preferred over the other. The open-sourced part of the dataset is publicly available to support further research in video quality assessment, content-adaptive streaming, and perceptual model development.

Figures (5)

• Figure 1: Sample frames from the source sequences.
• Figure 2: Spatial Information (SI) plotted against Temporal Information (TI) and Spatial Information (SI) plotted against Colorfulness (CF) of the source videos except the anchor contents.
• Figure 3: Min, max, and mean luminance metrics measured on all of the source sequences except the anchor contents in the database.
• Figure 4: Screenshot of the rating screen used to determine which video sequence has higher perceived quality.
• Figure 5: Examples Rate-Distortion Curve of four contents collected in our subjective study.