CISOL: An Open and Extensible Dataset for Table Structure Recognition in the Construction Industry

TL;DR

CISOL addresses reproducibility and domain-specific benchmarking for table structure recognition by introducing a transparent, open dataset built from anonymized, real-world civil engineering documents. It combines visual table component detection with simple heuristics to recover logical structure, and provides extensive metadata, an annotation workflow, and an open evaluation server. The dataset comprises over 120,000 annotated instances across 800+ images, with 3,288 German documents collected from 24 projects, licensed under CC BY 4.0. Findings show competitive benchmarking results (e.g., $mAP$ of 67.22 with YOLOv8), underscoring CISOL's utility for advancing TSR/TD in niche industrial domains and its extensibility for future expansions.

Abstract

Reproducibility and replicability are critical pillars of empirical research, particularly in machine learning, where they depend not only on the availability of models, but also on the datasets used to train and evaluate those models. In this paper, we introduce the Construction Industry Steel Ordering List (CISOL) dataset, which was developed with a focus on transparency to ensure reproducibility, replicability, and extensibility. CISOL provides a valuable new research resource and highlights the importance of having diverse datasets, even in niche application domains such as table extraction in civil engineering. CISOL is unique in that it contains real-world civil engineering documents from industry, making it a distinctive contribution to the field. The dataset contains more than 120,000 annotated instances in over 800 document images, positioning it as a medium-sized dataset that provides a robust foundation for Table Structure Recognition (TSR) and Table Detection (TD) tasks. Benchmarking results show that CISOL achieves 67.22 mAP@0.5:0.95:0.05 using the YOLOv8 model, outperforming the TSR-specific TATR model. This highlights the effectiveness of CISOL as a benchmark for advancing TSR, especially in specialized domains.

Figures (6)

• Figure 1: Ground truth annotations of the CISOL dataset on two example document images. The left side shows the table detection task (brown). The right side shows for each of the three images from top to bottom 1) spanning cells (light blue), 2) columns (orange) and 3) rows (beige) and headers (navy) locations. The variety in the table structure and distracting elements such as technical drawings of steel bending techniques show the challenges of the CISOL dataset.
• Figure 2: Further visualization of different types of layouts, using the same color coding as Figure \ref{['fig:teaser_graphic']}. The example on the left shows embedded tables within the actual table, which are considered to be particularly difficult examples. Embedded tables are not annotated in the CISOL dataset and are a prime candidate when extending the dataset. The middle image shows the large variations in the extent to which spanning cells can extend, while the right image shows a case where some spanning cells extend over embedded tables.
• Figure 3: On the left, the number of instances and images for each of the datasets are compared on a logarithmic scale, showing that CISOL is a medium sized dataset. The middle plot shows the class ratio for the five different datasets, a lower number of rows would indicate wider tables, while a lower number of columns would indicate narrower tables. The number of cells also gives an idea of the size of the tables. In the right plot, the density of annotations per document image can be analyzed, providing information on the variation within the dataset. For CISOL, while a large number of tables have a similar number of instances, there are a large number of outliers at the lower and upper ends. Some outliers exist beyond 1000 instances, but are not plotted to keep the plot zoomed in.
• Figure 4: Annotation pipeline of CISOL following four main stages, including two iterative approaches to guideline creation and the annotation campaign. The use of publicly available tools CVAT_ai_Corporation_Computer_Vision_Annotation_2023 and a straightforward, simple annotation pipeline makes the process easy to replicate and reproduce.
• Figure 5: Overview of data origin (left), selection for annotation (center), and dataset splits (right). The left chart shows data sources anonymized by company abbreviation, with 'Others' representing a mix of smaller contributors. Annotations have been prioritized for rare instances, with the remaining samples randomly selected. The middle chart compares the distribution of metadata tags between annotated and unannotated data, illustrating the balance of types and sizes. Finally, the right chart shows the split of annotated data into training, validation, and test sets.