Enginuity: Building an Open Multi-Domain Dataset of Complex Engineering Diagrams

TL;DR

The paper addresses the lack of open, multi-domain engineering diagram datasets with rich structural annotations, which hampers AI-driven diagram understanding and reasoning. It proposes Enginuity, a 50K+ automotive-diagram dataset with hierarchical relationships, spatial connections, and standardized ontologies to support tasks like component detection, relationship extraction, and diagram VQA, with plans for a CVPR 2026 workshop and leaderboard. A four-stage human-in-the-loop annotation pipeline, ontology alignment, and a four-way data split—including a withheld test set for domain shift—are introduced to ensure scalable, high-quality data and fair benchmarking. The work aims to accelerate AI-enabled scientific discovery and Industry 4.0 applications by enabling automated diagram parsing, digital-twin alignment, and cross-domain knowledge transfer.

Abstract

We propose Enginuity - the first open, large-scale, multi-domain engineering diagram dataset with comprehensive structural annotations designed for automated diagram parsing. By capturing hierarchical component relationships, connections, and semantic elements across diverse engineering domains, our proposed dataset would enable multimodal large language models to address critical downstream tasks including structured diagram parsing, cross-modal information retrieval, and AI-assisted engineering simulation. Enginuity would be transformative for AI for Scientific Discovery by enabling artificial intelligence systems to comprehend and manipulate the visual-structural knowledge embedded in engineering diagrams, breaking down a fundamental barrier that currently prevents AI from fully participating in scientific workflows where diagram interpretation, technical drawing analysis, and visual reasoning are essential for hypothesis generation, experimental design, and discovery.