Text2Traffic: A Text-to-Image Generation and Editing Method for Traffic Scenes

TL;DR

This work tackles the challenge of generating and editing realistic traffic-scene imagery from textual prompts by proposing Text2Traffic, a unified framework that uses controllable mask regions to jointly handle generation and editing. It combines a multi-view (vehicle-side and roadside) dataset with a two-stage training regime to improve text-image alignment and visual fidelity, and introduces a restoration-based fine-tuning approach with a region-weighted loss to emphasize small but critical elements. A prompt-aligned, LLM-assisted refinement further enhances semantic coherence. Experiments show strong performance in both generation and editing across perspectives, offering a practical pathway for generating diverse, high-quality traffic scene data for intelligent transportation applications.

Abstract

With the rapid advancement of intelligent transportation systems, text-driven image generation and editing techniques have demonstrated significant potential in providing rich, controllable visual scene data for applications such as traffic monitoring and autonomous driving. However, several challenges remain, including insufficient semantic richness of generated traffic elements, limited camera viewpoints, low visual fidelity of synthesized images, and poor alignment between textual descriptions and generated content. To address these issues, we propose a unified text-driven framework for both image generation and editing, leveraging a controllable mask mechanism to seamlessly integrate the two tasks. Furthermore, we incorporate both vehicle-side and roadside multi-view data to enhance the geometric diversity of traffic scenes. Our training strategy follows a two-stage paradigm: first, we perform conceptual learning using large-scale coarse-grained text-image data; then, we fine-tune with fine-grained descriptive data to enhance text-image alignment and detail quality. Additionally, we introduce a mask-region-weighted loss that dynamically emphasizes small yet critical regions during training, thereby substantially enhancing the generation fidelity of small-scale traffic elements. Extensive experiments demonstrate that our method achieves leading performance in text-based image generation and editing within traffic scenes.

Figures (7)

• Figure 1: Comparison of text-to-image performance for vehicle-side cameras across different models (key descriptions highlighted in yellow)
• Figure 2: Comparison of text-to-image performance for roadside cameras across different models.
• Figure 3: Process of image description generation method.
• Figure 4: Our restoration-based supervised fine-tuning method.
• Figure 5: Model architecture details.