UniVector: Unified Vector Extraction via Instance-Geometry Interaction

TL;DR

This work addresses the limitation of existing vector extraction methods that are specialized to a single vector structure by proposing UniVector, a unified VE framework that encodes vectors as structured queries and refines them through an instance–geometry interaction. The approach combines a unified vector encoding, an instance–geometry interactive decoder, and a Dynamic Shape Constraint to jointly capture global topology and local geometry across polygons, polylines, and line segments. A new Multi-Vector dataset is introduced to benchmark multi-structure VE, and comprehensive experiments show state-of-the-art results on both single-structure and multi-structure VE tasks, with notable efficiency gains. The work lays groundwork for a cross-structure VE foundation and promises impactful downstream applications in mapping, robotics, and digital design.

Abstract

Vector extraction retrieves structured vector geometry from raster images, offering high-fidelity representation and broad applicability. Existing methods, however, are usually tailored to a single vector type (e.g., polygons, polylines, line segments), requiring separate models for different structures. This stems from treating instance attributes (category, structure) and geometric attributes (point coordinates, connections) independently, limiting the ability to capture complex structures. Inspired by the human brain's simultaneous use of semantic and spatial interactions in visual perception, we propose UniVector, a unified VE framework that leverages instance-geometry interaction to extract multiple vector types within a single model. UniVector encodes vectors as structured queries containing both instance- and geometry-level information, and iteratively updates them through an interaction module for cross-level context exchange. A dynamic shape constraint further refines global structures and key points. To benchmark multi-structure scenarios, we introduce the Multi-Vector dataset with diverse polygons, polylines, and line segments. Experiments show UniVector sets a new state of the art on both single- and multi-structure VE tasks. Code and dataset will be released at https://github.com/yyyyll0ss/UniVector.

Figures (5)

• Figure 1: a. Compared with raster images, vector data are lightweight, high-fidelity, and easily editable, and are widely used in graphic design, geographic cartography, and autonomous driving. b. Comparison of specific vs. unified vector extraction: while prior models xue2023holisticallyzorzi2022polyworldgjiao2024polyrcnnhu2024polyroadxu2021icurbxu2021line handle only one vector structure, UniVector extracts multiple structures within a single framework.
• Figure 2: Comparison of previous frameworks and our UniVector. Existing methods zorzi2022polyworldg, xu2022accurate, jiao2024polyrcnn usually split vector extraction into two cascaded tasks, often causing shape inaccuracies or topological errors. Inspired by the human brain’s simultaneous use of semantic and spatial interactions in visual perception, UniVector models instance–geometry interaction to capture both precise shapes and topology across diverse structures.
• Figure 3: Overview of UniVector.a. The pipeline includes unified vector encoding, instance-geometry interactive decoding, and a dynamic shape constraint. b. Vector instances and geometric attributes are first encoded as unified queries for interactive learning. c. Instance-geometry interaction then iteratively refines these queries for cross-level learning. d. A dynamic shape constraint ensures global structural consistency and local accuracy.
• Figure 4: a. The process of encoding vectors into structured queries. b. Comparison of query interaction: the global interaction at geometric-level yue2023connecting; the decoupled interaction at geometric-level liao2024maptrv2; and our structured instance-geometry interaction. c. Motivation of Dynamic Shape Constraints (DSC). Fixed-order constraints risk keypoint misalignment, while DSC dynamically matches predictions to ground truth for optimal keypoint pairing.
• Figure 5: Visualization of UniVector on different datasets, including polygons, polylines, line segments, and the simultaneous extraction of all three.