Eevee: Towards Close-up High-resolution Video-based Virtual Try-on

TL;DR

This work addresses the limitations of current video-based virtual try-on by introducing Eevee, a high-resolution dataset that provides both full-shot and close-up videos along with rich garment detail images and textual descriptions. It proposes VGID, a dedicated metric for garment consistency in close-ups, and demonstrates that fine-tuning a diffusion-transformer model (VACE) with LoRA and detailed garment inputs yields superior texture fidelity compared to existing methods. Through extensive quantitative and qualitative evaluations on Eevee, the authors establish strong baselines and reveal texture- and structure-preservation gaps in prior approaches. The dataset and metric hold practical value for fashion marketing, enabling more realistic and informative close-up visualizations of garments in video try-on tasks.

Abstract

Video virtual try-on technology provides a cost-effective solution for creating marketing videos in fashion e-commerce. However, its practical adoption is hindered by two critical limitations. First, the reliance on a single garment image as input in current virtual try-on datasets limits the accurate capture of realistic texture details. Second, most existing methods focus solely on generating full-shot virtual try-on videos, neglecting the business's demand for videos that also provide detailed close-ups. To address these challenges, we introduce a high-resolution dataset for video-based virtual try-on. This dataset offers two key features. First, it provides more detailed information on the garments, which includes high-fidelity images with detailed close-ups and textual descriptions; Second, it uniquely includes full-shot and close-up try-on videos of real human models. Furthermore, accurately assessing consistency becomes significantly more critical for the close-up videos, which demand high-fidelity preservation of garment details. To facilitate such fine-grained evaluation, we propose a new garment consistency metric VGID (Video Garment Inception Distance) that quantifies the preservation of both texture and structure. Our experiments validate these contributions. We demonstrate that by utilizing the detailed images from our dataset, existing video generation models can extract and incorporate texture features, significantly enhancing the realism and detail fidelity of virtual try-on results. Furthermore, we conduct a comprehensive benchmark of recent models. The benchmark effectively identifies the texture and structural preservation problems among current methods.

Figures (6)

• Figure 1: An illustrative sample from our proposed virtual try-on dataset. Compared to existing datasets, ours provides a richer collection of data, including both full-shot and close-up videos of the person, detailed garment images, and rich textual descriptions. Furthermore, all images and videos are provided at a higher resolution. More examples can be found in the Appendix.
• Figure 2: A comparison of garment image and video frame, along with their zoomed-in details, from the ViViD dataset shown on top and the Eevee dataset shown on bottom.
• Figure 3: The overall fine-tuning pipeline of the VACE on our dataset. Compared to other video-based virtual try-on models, our VACE fine-tuning process includes an additional step where detailed garment images are fed into the network through the Wan VAE Encoder.
• Figure 4: VBench evaluation results of video-based virtual try-on models. We visualize the evaluation results of four models in 4 VBench dimensions under both paired and unpaired settings. We normalize the results per dimension for clearer comparisons. For comprehensive numerical results, please refer to the Appendix.
• Figure 5: Qualitative comparison of close-up video virtual try-on results on the Eevee dataset. Our fine-tuned VACE model not only accurately reproduces the patterns of the input garment, but the generated video is also more realistic overall. Please zoom in for better visualization.