A Formalization of Image Vectorization by Region Merging

TL;DR

It is remarked that image vectorization is nothing but an image segmentation, and that it can be built by fine to coarse region merging, and that the curve smoothing, implicit in all vectorization methods, can be performed by the shape-preserving affine scale space.

Abstract

Image vectorization converts raster images into vector graphics composed of regions separated by curves. Typical vectorization methods first define the regions by grouping similar colored regions via color quantization, then approximate their boundaries by Bezier curves. In that way, the raster input is converted into an SVG format parameterizing the regions' colors and the Bezier control points. This compact representation has many graphical applications thanks to its universality and resolution-independence. In this paper, we remark that image vectorization is nothing but an image segmentation, and that it can be built by fine to coarse region merging. Our analysis of the problem leads us to propose a vectorization method alternating region merging and curve smoothing. We formalize the method by alternate operations on the dual and primal graph induced from any domain partition. In that way, we address a limitation of current vectorization methods, which separate the update of regional information from curve approximation. We formalize region merging methods by associating them with various gain functionals, including the classic Beaulieu-Goldberg and Mumford-Shah functionals. More generally, we introduce and compare region merging criteria involving region number, scale, area, and internal standard deviation. We also show that the curve smoothing, implicit in all vectorization methods, can be performed by the shape-preserving affine scale space. We extend this flow to a network of curves and give a sufficient condition for the topological preservation of the segmentation. The general vectorization method that follows from this analysis shows explainable behaviors, explicitly controlled by a few intuitive parameters. It is experimentally compared to state-of-the-art software and proved to have comparable or superior fidelity and cost efficiency.

Figures (18)

• Figure 1: Number of colors and region complexity. (a) Raster image of size $500\times292$ (painting by Forrest Bess). (b) The proposed Area region merging \ref{['eq_area_gain']} gives $N=151$ regions, yielding simpler boundaries and flatter regions. (c) Color quantization by k-means with $k=5$ colors. This results in $2418$ connected components with complicated boundaries. Region merging (b) reduces the geometric complexity, which is more suitable for efficient vectorization; whereas quantization (c) reduces complexity only in the color space.
• Figure 2: Primal and dual graph induced from image partition. (a) Region partition $\mathcal{P}$ of an image $f$. (b) The dual graph represents each region $O_i$ as a node, and two nodes are connected by an edge if the corresponding regions are adjacent. (c) The primal graph represents each junction $\mathbf{P}_k$ as a node, and two nodes are connected by boundary curves $\mathcal{C}_m$. The dual graph reflects the adjacency relations among regions, and the primal graph shows the geometry of the partitioning boundaries.
• Figure 3: The dual graph update. From the given partition in (a), (b) illustrates when the yellow rectangle and the purple ellipse are merged. One node from the dual graph is removed as well as the associated edges. After one primal step, in (c), the dark region is merged with the background; and after another one, in (d), the yellow disk is merged with the background, yielding only two regions in the image domain. The additional element $O_0$ (the circle node) denotes the out-of-domain region.
• Figure 4: The primal graph update. After dual steps, the raster image in (a) induces a primal graph with pixelated boundaries in (b), i.e., $T=0.0$. The boundary curves are smoothed by the affine shortening flow \ref{['eq_as_flow']} with the evolution time (c) $T=0.2$, (d) $T=1.0$, and (e) $T=1.5$. The black dot specifies the junction point, which is fixed during the primal update.
• Figure 5: Region merging comparison. (a) The input image contains $8$ regions. We merge regions until there are $N=7$ (first row), and $N=5$ (second row) using (b) Area region merging \ref{['eq_area_gain']}, (c) BG region merging \ref{['eq_BG_gain']}, (d) Scale region merging \ref{['eq_scale_gain']}, and (e) MS region merging \ref{['eq_MS_gain']}, respectively. Among these methods, Area tends to find larger regions regardless of their perimeters, BG follows a merging order that is independent of shapes, Scale favors large and convex region, and MS prefers bright shapes with shorter perimeters.

Theorems & Definitions (18)

• Proposition 3.1
• proof
• Remark 3.1
• Remark 3.2
• Proposition 3.2
• proof
• Theorem 3.3: Maximal number of regions
• proof
• Proposition 3.3: Elimination of small regions
• Corollary 3.1: Lower bound on perimeter