Symbol as Points: Panoptic Symbol Spotting via Point-based Representation

TL;DR

This work tackles panoptic symbol spotting in CAD drawings by treating graphic primitives as a set of 2D points and applying point-cloud–style segmentation. It introduces SymPoint, which combines a Point Transformer backbone with a Mask2Former–like spotting head and adds Attention with Connection Module (ACM), Contrastive Connection Learning (CCL), and KNN interpolation to handle primitive-level masks and local primitive connections. On FloorPlanCAD, SymPoint achieves state-of-the-art panoptic symbol spotting performance, including substantial improvements in PQ and RQ over prior methods, while offering faster inference by avoiding rasterization. Comprehensive ablations validate the value of ACM, CCL, and KNN interpolation, and the method shows good generalization to other vector-graphics datasets. This work advances CAD symbol spotting by leveraging a scalable, point-based representation that aligns with the intrinsic structure of vector graphics.

Abstract

This work studies the problem of panoptic symbol spotting, which is to spot and parse both countable object instances (windows, doors, tables, etc.) and uncountable stuff (wall, railing, etc.) from computer-aided design (CAD) drawings. Existing methods typically involve either rasterizing the vector graphics into images and using image-based methods for symbol spotting, or directly building graphs and using graph neural networks for symbol recognition. In this paper, we take a different approach, which treats graphic primitives as a set of 2D points that are locally connected and use point cloud segmentation methods to tackle it. Specifically, we utilize a point transformer to extract the primitive features and append a mask2former-like spotting head to predict the final output. To better use the local connection information of primitives and enhance their discriminability, we further propose the attention with connection module (ACM) and contrastive connection learning scheme (CCL). Finally, we propose a KNN interpolation mechanism for the mask attention module of the spotting head to better handle primitive mask downsampling, which is primitive-level in contrast to pixel-level for the image. Our approach, named SymPoint, is simple yet effective, outperforming recent state-of-the-art method GAT-CADNet by an absolute increase of 9.6% PQ and 10.4% RQ on the FloorPlanCAD dataset. The source code and models will be available at https://github.com/nicehuster/SymPoint.

Figures (11)

• Figure 1: Illustration of constructing point-based representation.
• Figure 2: The overview of our method. After transfering CAD drawings to primitive points, we use a backbone to extract multi-resolution features $F_r$ and append a symbol spotting head to spot and recognize symbols. During this process, we propose attention with connection module(ACM), which utilizes primitive connection information when performing self-attention in the first stage of backone. Subsequently, we propose contrastive connection learning(CCL) to enhance the discriminability between connected primitive features. Finally, we propose KNN interpolation for attention mask downsampling(AMD) to effetively downsample the high-resolution attention masks.
• Figure 3: (a) Set of primitives and its connection, primitives are disintegrated for clarity. (b) Locally connected primitives are considered in the attention layers. (c) Locally connected primitives do not always belong to the same category.
• Figure 4: Qualitative comparison of panoptic symbol spotting results with CADTransformer. Primitives belonging to different classes are represented in distinct colors. The colormap for each category can be referenced in \ref{['fig:color_map']}.
• Figure 5: Convergence curves with/without the ACM Module on SESYD-floorplans.