Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On

Xiaoye Liang, Zhiyuan Qu, Mingye Zou, Jiaxin Liu, Lai Jiang, Mai Xu, Yiheng Zhu

Abstract

As virtual try-on (VTON) continues to advance, a growing number of real-world scenarios have emerged, pushing beyond the ability of the existing specialized VTON models. Meanwhile, universal multi-reference image editing models have progressed rapidly and exhibit strong generalization in visual editing, suggesting a promising route toward more flexible VTON systems. However, despite their strong capabilities, the strengths and limitations of universal editors for VTON remain insufficiently explored due to the lack of systematic evaluation benchmarks. To address this gap, we introduce VTEdit-Bench, a comprehensive benchmark designed to evaluate universal multi-reference image editing models across various realistic VTON scenarios. VTEdit-Bench contains 24,220 test image pairs spanning five representative VTON tasks with progressively increasing complexity, enabling systematic analysis of robustness and generalization. We further propose VTEdit-QA, a reference-aware VLM-based evaluator that assesses VTON performance from three key aspects: model consistency, cloth consistency, and overall image quality. Through this framework, we systematically evaluate eight universal editing models and compare them with seven specialized VTON models. Results show that top universal editors are competitive on conventional tasks and generalize more stably to harder scenarios, but remain challenged by complex reference configurations, particularly multi-cloth conditioning.

VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On

Abstract

As virtual try-on (VTON) continues to advance, a growing number of real-world scenarios have emerged, pushing beyond the ability of the existing specialized VTON models. Meanwhile, universal multi-reference image editing models have progressed rapidly and exhibit strong generalization in visual editing, suggesting a promising route toward more flexible VTON systems. However, despite their strong capabilities, the strengths and limitations of universal editors for VTON remain insufficiently explored due to the lack of systematic evaluation benchmarks. To address this gap, we introduce VTEdit-Bench, a comprehensive benchmark designed to evaluate universal multi-reference image editing models across various realistic VTON scenarios. VTEdit-Bench contains 24,220 test image pairs spanning five representative VTON tasks with progressively increasing complexity, enabling systematic analysis of robustness and generalization. We further propose VTEdit-QA, a reference-aware VLM-based evaluator that assesses VTON performance from three key aspects: model consistency, cloth consistency, and overall image quality. Through this framework, we systematically evaluate eight universal editing models and compare them with seven specialized VTON models. Results show that top universal editors are competitive on conventional tasks and generalize more stably to harder scenarios, but remain challenged by complex reference configurations, particularly multi-cloth conditioning.
Paper Structure (49 sections, 2 equations, 11 figures, 6 tables)

This paper contains 49 sections, 2 equations, 11 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Image-based VTON
  4. Instruction-based Image Editing Model
  5. Image Editing Benchmark
  6. VTEdit-Bench
  7. Task Description
  8. Shop2Model
  9. Shop2MultiModel
  10. Shop2MultiView
  11. Model2Model
  12. MultiShop2Model
  13. Auxiliary conditions.
  14. Unified Evaluation Metrics
  15. Model Consistency
  16. ...and 34 more sections

Figures (11)

  • Figure 1: Overview of VTEdit-Bench, illustrating VTON tasks and dataset composition, the comparison between universal and specialized models, and the proposed unified evaluation metrics.
  • Figure 2: Illustration of the five tasks in VTEdit-Bench, showing the diversity of shop sources and model sources in the dataset, as well as how different tasks are constructed and connected through shared input compositions.
  • Figure 3: The annotation pipeline for capturing human preferences.
  • Figure 4: Overview of the VTEdit-QA framework for VLM-based evaluation of VTON outputs.
  • Figure 5: SRCC between human annotators and GPT-4o under different scoring settings.
  • ...and 6 more figures