Kyrtos: A methodology for automatic deep analysis of graphic charts with curves in technical documents

TL;DR

Kyrtos tackles automatic deep understanding of technical documents by focusing on charts with curves. It combines a chart recognition module that extracts 2D middle-points via unevenness detection and hierarchical clustering with a chart analysis module that uses the Kyrtos formal language to derive relations, convert them to attributed graphs and natural language, and ultimately generate SPN representations for curve functionality. The work introduces the Kyrtos alphabet, grammar, and operators, enabling precise mapping from visual relations to SPN kernels and facilitating curve reconstruction. Evaluation on synthetic datasets shows high reconstruction fidelity and provides insight into the trade-offs between merging rules and computational cost, while noting current limitations to multi-color curves and future directions including bar charts, curves of the same color, and damaged-document scenarios.

Abstract

Deep Understanding of Technical Documents (DUTD) has become a very attractive field with great potential due to large amounts of accumulated documents and the valuable knowledge contained in them. In addition, the holistic understanding of technical documents depends on the accurate analysis of its particular modalities, such as graphics, tables, diagrams, text, etc. and their associations. In this paper, we introduce the Kyrtos methodology for the automatic recognition and analysis of charts with curves in graphics images of technical documents. The recognition processing part adopts a clustering based approach to recognize middle-points that delimit the line-segments that construct the illustrated curves. The analysis processing part parses the extracted line-segments of curves to capture behavioral features such as direction, trend and etc. These associations assist the conversion of recognized segments' relations into attributed graphs, for the preservation of the curves' structural characteristics. The graph relations are also are expressed into natural language (NL) text sentences, enriching the document's text and facilitating their conversion into Stochastic Petri-net (SPN) graphs, which depict the internal functionality represented in the chart image. Extensive evaluation results demonstrate the accuracy of Kyrtos' recognition and analysis methods by measuring the structural similarity between input chart curves and the approximations generated by Kyrtos for charts with multiple functions.

Figures (17)

• Figure 1: Pipeline of the Kyrtos chart analysis methodology.
• Figure 2: Mixed curves of different colors (left), partitioned curves of the same color (right) and bars chart (bottom).
• Figure 3: The Chart Recognition Methodology of Kyrtos.
• Figure 4: Example of recognizing changes in direction (unevennesses) using the slope equation and the unevenness criteria component.
• Figure 5: Example of clustering detected unevenness points for threshold of 4 pixel distance (left column) and of 3 pixel distance (right column).