Generative Digital Twins: Vision-Language Simulation Models for Executable Industrial Systems

TL;DR

The paper tackles automating industrial digital-twin authoring by unifying vision and language to generate executable FlexScript from sketches and prompts. It introduces the GDT dataset (≈120K prompt–sketch–code triplets) and three metrics (SVR, PMR, ESR) to evaluate structure, parameters, and executability. It proposes the Vision-Language Simulation Model (VLSM), a multimodal framework that fuses vision encoders with code-pretrained LLM backbones, with extensive ablations to identify effective configurations. Results show near-perfect structural accuracy and strong execution robustness, establishing a foundation for scalable, automation-friendly simulation in industrial contexts.

Abstract

We propose a Vision-Language Simulation Model (VLSM) that unifies visual and textual understanding to synthesize executable FlexScript from layout sketches and natural-language prompts, enabling cross-modal reasoning for industrial simulation systems. To support this new paradigm, the study constructs the first large-scale dataset for generative digital twins, comprising over 120,000 prompt-sketch-code triplets that enable multimodal learning between textual descriptions, spatial structures, and simulation logic. In parallel, three novel evaluation metrics, Structural Validity Rate (SVR), Parameter Match Rate (PMR), and Execution Success Rate (ESR), are proposed specifically for this task to comprehensively evaluate structural integrity, parameter fidelity, and simulator executability. Through systematic ablation across vision encoders, connectors, and code-pretrained language backbones, the proposed models achieve near-perfect structural accuracy and high execution robustness. This work establishes a foundation for generative digital twins that integrate visual reasoning and language understanding into executable industrial simulation systems.

Figures (9)

• Figure 1: Overview of the generative digital twins framework, showing major challenges in digital-twin modeling, the construction of the GDT-120K dataset with evaluation metrics, and the Vision-Language Simulation Models (VLSM) workflow.
• Figure 2: Workflow of the GDT-120K dataset construction, integrating curated factory data, statistical validation, and FlexSim instantiation with human–AI co-authored prompts to create aligned multimodal pairs for model training and evaluation.
• Figure 3: Dataset statistics across layout type, automation level, industry and layout categories.
• Figure 4: Overall architecture of the VLSM, integrating vision and language modules to generate executable FlexScript evaluated by SVR, PMR, and ESR (with BLEU as a supplementary metric).
• Figure 5: LLM Evaluation across Epochs. Performance of seven LLM baselines over 10 epochs on four metrics.