RoboSVG: A Unified Framework for Interactive SVG Generation with Multi-modal Guidance

TL;DR

RoboSVG introduces a unified multimodal framework for generating interactive SVGs guided by text, images, and partial SVGs, addressing the challenge of producing semantically accurate and structurally valid vector graphics for robot drawing. It builds RoboDraw, a 1-million-sample dataset with complete, partial SVGs and textual descriptions to support four generation tasks, enabling robust supervised and interactive learning. The model comprises a Guidance Generator that fuses visual, textual, and numerical cues with dedicated SVG generators, achieving state-of-the-art performance on both basic (Text-to-SVG, Image-to-SVG) and interactive (PartialSVG-to-SVG, PartialImage-to-SVG) tasks across RoboDraw and SVGenius benchmarks. The results highlight the benefits of multimodal guidance and specialized generation modules, with practical implications for versatile SVG generation in design and robotics, and the dataset and code will be released publicly soon.

Abstract

Scalable Vector Graphics (SVGs) are fundamental to digital design and robot control, encoding not only visual structure but also motion paths in interactive drawings. In this work, we introduce RoboSVG, a unified multimodal framework for generating interactive SVGs guided by textual, visual, and numerical signals. Given an input query, the RoboSVG model first produces multimodal guidance, then synthesizes candidate SVGs through dedicated generation modules, and finally refines them under numerical guidance to yield high-quality outputs. To support this framework, we construct RoboDraw, a large-scale dataset of one million examples, each pairing an SVG generation condition (e.g., text, image, and partial SVG) with its corresponding ground-truth SVG code. RoboDraw dataset enables systematic study of four tasks, including basic generation (Text-to-SVG, Image-to-SVG) and interactive generation (PartialSVG-to-SVG, PartialImage-to-SVG). Extensive experiments demonstrate that RoboSVG achieves superior query compliance and visual fidelity across tasks, establishing a new state of the art in versatile SVG generation. The dataset and source code of this project will be publicly available soon.

Figures (5)

• Figure 1: The four tasks introduced in our RoboDraw dataset, including basic generation (i.e., Text-to-SVG and Image-to-SVG) and interactive generation (i.e., PartialSVG-to-SVG and PartialImage-to-SVG).
• Figure 2: The architecture of the RoboSVG model. The left part illustrates the architecture of RoboSVG, consisting of the guidance generator and the SVG generator. The right part shows the workflow: the guidance generator transforms the input query into multimodal guidance, the SVG generator produces candidate SVGs, and the final output SVG is selected from these candidates.
• Figure 3: Qualitative results of different models on the RoboDraw test split. We illustrate four SVG generation tasks, with two examples shown for each task.
• Figure 4: The interface example of the user study. Here, we present an example of the PartialImage-to-SVG task, where the input query consists of the input prompt and PartialImage, and the generated SVGs are produced by three different methods (Qwen-2.5-VL-72B, GPT-4o, and our RoboSVG).
• Figure 5: Qualitative results of RoboSVG on four SVG generation tasks.