Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CHUG: Crowdsourced User-Generated HDR Video Quality Dataset

Shreshth Saini, Alan C. Bovik, Neil Birkbeck, Yilin Wang, Balu Adsumilli

TL;DR

CHUG addresses the lack of large-scale, real-world UGC-HDR quality datasets for HDR-VQA by introducing 856 UGC-HDR source videos transformed into 5,992 clips and rated with 211,848 subjective judgments collected via AMT. The study employs a bitrate ladder to simulate social-media streaming and uses the $SUREAL$ method to compute $MOS$ while accounting for subject bias and inconsistency, enabling robust no-reference HDR-VQA benchmarking. Analyses reveal how spatial-temporal complexity, orientation, and bitrate-resolution interact with perceived quality, and CHUG is shown to offer broader distortion realism than existing HDR datasets like LIVE-HDR and SFV+HDR. The dataset and scores are publicly available to spur NR-VQA model development and improve real-world HDR streaming QA.

Abstract

High Dynamic Range (HDR) videos enhance visual experiences with superior brightness, contrast, and color depth. The surge of User-Generated Content (UGC) on platforms like YouTube and TikTok introduces unique challenges for HDR video quality assessment (VQA) due to diverse capture conditions, editing artifacts, and compression distortions. Existing HDR-VQA datasets primarily focus on professionally generated content (PGC), leaving a gap in understanding real-world UGC-HDR degradations. To address this, we introduce CHUG: Crowdsourced User-Generated HDR Video Quality Dataset, the first large-scale subjective study on UGC-HDR quality. CHUG comprises 856 UGC-HDR source videos, transcoded across multiple resolutions and bitrates to simulate real-world scenarios, totaling 5,992 videos. A large-scale study via Amazon Mechanical Turk collected 211,848 perceptual ratings. CHUG provides a benchmark for analyzing UGC-specific distortions in HDR videos. We anticipate CHUG will advance No-Reference (NR) HDR-VQA research by offering a large-scale, diverse, and real-world UGC dataset. The dataset is publicly available at: https://shreshthsaini.github.io/CHUG/.

CHUG: Crowdsourced User-Generated HDR Video Quality Dataset

TL;DR

CHUG addresses the lack of large-scale, real-world UGC-HDR quality datasets for HDR-VQA by introducing 856 UGC-HDR source videos transformed into 5,992 clips and rated with 211,848 subjective judgments collected via AMT. The study employs a bitrate ladder to simulate social-media streaming and uses the method to compute while accounting for subject bias and inconsistency, enabling robust no-reference HDR-VQA benchmarking. Analyses reveal how spatial-temporal complexity, orientation, and bitrate-resolution interact with perceived quality, and CHUG is shown to offer broader distortion realism than existing HDR datasets like LIVE-HDR and SFV+HDR. The dataset and scores are publicly available to spur NR-VQA model development and improve real-world HDR streaming QA.

Abstract

High Dynamic Range (HDR) videos enhance visual experiences with superior brightness, contrast, and color depth. The surge of User-Generated Content (UGC) on platforms like YouTube and TikTok introduces unique challenges for HDR video quality assessment (VQA) due to diverse capture conditions, editing artifacts, and compression distortions. Existing HDR-VQA datasets primarily focus on professionally generated content (PGC), leaving a gap in understanding real-world UGC-HDR degradations. To address this, we introduce CHUG: Crowdsourced User-Generated HDR Video Quality Dataset, the first large-scale subjective study on UGC-HDR quality. CHUG comprises 856 UGC-HDR source videos, transcoded across multiple resolutions and bitrates to simulate real-world scenarios, totaling 5,992 videos. A large-scale study via Amazon Mechanical Turk collected 211,848 perceptual ratings. CHUG provides a benchmark for analyzing UGC-specific distortions in HDR videos. We anticipate CHUG will advance No-Reference (NR) HDR-VQA research by offering a large-scale, diverse, and real-world UGC dataset. The dataset is publicly available at: https://shreshthsaini.github.io/CHUG/.

Paper Structure

This paper contains 15 sections, 1 equation, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Crowdsourcing UGC-HDR Videos
  3. Source Video Collection and UGC Diversity
  4. Bitrate Ladder for Realistic Streaming Simulation
  5. Details of Subjective Study
  6. Study Design and Rating Procedure
  7. Participant Screening and Quality Control
  8. Data Analysis
  9. Processing of Subjective Scores
  10. Challenging Content and High Variance MOS
  11. Spatial-Temporal Features vs. MOS
  12. Effect of Orientation on MOS
  13. MOS Across Resolution and Bitrate
  14. Comparing CHUG with Existing HDR Datasets
  15. Conclusion

Figures (10)

  • Figure 1: Sample frames from the CHUG dataset, showcasing diverse real-world UGC-HDR content with variations in lighting, motion, orientation, and distortions. Best viewed when zoomed in.
  • Figure 2: Resolution distribution of CHUG dataset, maintaining a balanced mix of landscape and portrait videos to study orientation-based perceptual differences.
  • Figure 3: Compression artifacts introduced via bitladder. Left: 1080p reference, Middle: 720p at 2 Mbps, Right: 360p at 0.2 Mbps.
  • Figure 4: Rating interface used for the AMT study (Best viewed zoomed in).
  • Figure 5: (a) MOS distribution of all videos; (b) Inter-subject correlation
  • ...and 5 more figures