Style Customization of Text-to-Vector Generation with Image Diffusion Priors

TL;DR

This work tackles the challenge of style-customized SVG generation from text prompts by integrating a two-stage pipeline that leverages both feed-forward T2V modeling and image diffusion priors. In the first stage, a path-level diffusion model learns content and structural regularity from black-and-white SVGs, ensuring coherent SVG layouts. In the second stage, style priors from customized T2I diffusion models are distilled to guide a T2V model, enabling diverse, high-quality SVGs in user-defined styles with a fast feed-forward workflow. The approach demonstrates strong results across vector, image, and text-level metrics, validated by extensive experiments, ablations, and a user study, while noting limitations related to semantic richness of the training SVG dataset and potential style detail loss for very complex references.

Abstract

Scalable Vector Graphics (SVGs) are highly favored by designers due to their resolution independence and well-organized layer structure. Although existing text-to-vector (T2V) generation methods can create SVGs from text prompts, they often overlook an important need in practical applications: style customization, which is vital for producing a collection of vector graphics with consistent visual appearance and coherent aesthetics. Extending existing T2V methods for style customization poses certain challenges. Optimization-based T2V models can utilize the priors of text-to-image (T2I) models for customization, but struggle with maintaining structural regularity. On the other hand, feed-forward T2V models can ensure structural regularity, yet they encounter difficulties in disentangling content and style due to limited SVG training data. To address these challenges, we propose a novel two-stage style customization pipeline for SVG generation, making use of the advantages of both feed-forward T2V models and T2I image priors. In the first stage, we train a T2V diffusion model with a path-level representation to ensure the structural regularity of SVGs while preserving diverse expressive capabilities. In the second stage, we customize the T2V diffusion model to different styles by distilling customized T2I models. By integrating these techniques, our pipeline can generate high-quality and diverse SVGs in custom styles based on text prompts in an efficient feed-forward manner. The effectiveness of our method has been validated through extensive experiments. The project page is https://customsvg.github.io.

Figures (10)

• Figure 1: Our two-stage style customization pipeline for SVGs. (a) In Stage 1, we train a path-level T2V diffusion model on black-and-white SVG datasets to focus on learning the contents and structures of SVGs. (b) In Stage 2, we learn various styles of SVGs by distilling priors from different customized T2I models. (c) After training, our T2V model can generate SVGs in custom styles learned during Stage 2 in a feed-forward manner by appending the corresponding style token to the text prompt. Exemplar SVGs are from ©SVGRepo.
• Figure 2: (a) SVG examples from the dataset. (b) SVG samples generated from random noise by our T2V diffusion model in Stage 1.
• Figure 3: Qualitative comparison with optimization-based T2V methods. Exemplar SVGs are from ©SVGRepo.
• Figure 4: Qualitative comparison to feed-forward T2V methods. Exemplar SVGs are from ©SVGRepo.
• Figure 5: User Study. We show the human preferences in %.