Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on

Xinyi Wei, Sijing Wu, Zitong Xu, Yunhao Li, Huiyu Duan, Xiongkuo Min, Guangtao Zhai

TL;DR

This work addresses the lack of perceptual quality evaluation for image-based virtual try-on (VTON) by introducing VTONQA, the first large-scale, multi-dimensional QA dataset tailored for VTON. It collects 8,132 VTON-generated images from 11 models and 24,396 MOS across clothing fit, body compatibility, and overall quality, accompanied by subjective annotations from 40 raters. The authors benchmark 11 VTON models and 17 IQA metrics, revealing that closed-source methods outperform open-source ones and that existing objective metrics poorly predict perceptual judgments, especially for clothing fit. The dataset and benchmarks enable perceptually aligned evaluation, guiding future improvements in VTON models and quality assessment methods for practical, real-world use.

Abstract

With the rapid development of e-commerce and digital fashion, image-based virtual try-on (VTON) has attracted increasing attention. However, existing VTON models often suffer from artifacts such as garment distortion and body inconsistency, highlighting the need for reliable quality evaluation of VTON-generated images. To this end, we construct VTONQA, the first multi-dimensional quality assessment dataset specifically designed for VTON, which contains 8,132 images generated by 11 representative VTON models, along with 24,396 mean opinion scores (MOSs) across three evaluation dimensions (i.e., clothing fit, body compatibility, and overall quality). Based on VTONQA, we benchmark both VTON models and a diverse set of image quality assessment (IQA) metrics, revealing the limitations of existing methods and highlighting the value of the proposed dataset. We believe that the VTONQA dataset and corresponding benchmarks will provide a solid foundation for perceptually aligned evaluation, benefiting both the development of quality assessment methods and the advancement of VTON models.

VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on

TL;DR

This work addresses the lack of perceptual quality evaluation for image-based virtual try-on (VTON) by introducing VTONQA, the first large-scale, multi-dimensional QA dataset tailored for VTON. It collects 8,132 VTON-generated images from 11 models and 24,396 MOS across clothing fit, body compatibility, and overall quality, accompanied by subjective annotations from 40 raters. The authors benchmark 11 VTON models and 17 IQA metrics, revealing that closed-source methods outperform open-source ones and that existing objective metrics poorly predict perceptual judgments, especially for clothing fit. The dataset and benchmarks enable perceptually aligned evaluation, guiding future improvements in VTON models and quality assessment methods for practical, real-world use.

Abstract

With the rapid development of e-commerce and digital fashion, image-based virtual try-on (VTON) has attracted increasing attention. However, existing VTON models often suffer from artifacts such as garment distortion and body inconsistency, highlighting the need for reliable quality evaluation of VTON-generated images. To this end, we construct VTONQA, the first multi-dimensional quality assessment dataset specifically designed for VTON, which contains 8,132 images generated by 11 representative VTON models, along with 24,396 mean opinion scores (MOSs) across three evaluation dimensions (i.e., clothing fit, body compatibility, and overall quality). Based on VTONQA, we benchmark both VTON models and a diverse set of image quality assessment (IQA) metrics, revealing the limitations of existing methods and highlighting the value of the proposed dataset. We believe that the VTONQA dataset and corresponding benchmarks will provide a solid foundation for perceptually aligned evaluation, benefiting both the development of quality assessment methods and the advancement of VTON models.
Paper Structure (15 sections, 1 equation, 8 figures, 3 tables)

This paper contains 15 sections, 1 equation, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Virtual Try-On Methods and Datasets
  4. Evaluation of Virtual Try-On Results
  5. Dataset and Evaluation Setup
  6. Dataset construction
  7. Virtual try-on algorithms
  8. Subjective Experiment
  9. Subjective Data Processing
  10. Multi-dimensional Analysis
  11. Analysis
  12. Basic Analysis
  13. Baseline Experiment
  14. Future work
  15. Conclusion

Figures (8)

  • Figure 1: Illustration of the image-based virtual try-on pipeline.
  • Figure 2: Examples for each clothing category in VTONQA.
  • Figure 3: Examples for each human body category in VTONQA.
  • Figure 4: Illustration of the GUI used in the subjective study.
  • Figure 5: Examples from the proposed VTONQA dataset. We illustrate poor (20–40), average (40–60), and good (60–80) cases for three evaluation dimensions: (a) clothing fit, (b) body compatibility, and (c) overall quality.
  • ...and 3 more figures