Semantic Line Combination Detector

TL;DR

This work tackles semantic line detection by moving beyond pairwise line harmony to holistic evaluation of line combinations. It introduces SLCD, which generates $K$ reliable lines from $N$ candidates, forms all $2^{K}$ line combinations, and scores them via a multi-branch pipeline that includes semantic feature grouping with cross-attention and a compositional feature extractor, followed by a regression head. Two novel losses guide region partitioning ($\mathcal{L}_{\rm SRS}$) and composition scoring ($\mathcal{L}_{\rm reg}, \mathcal{L}_{\rm rank}$), and a new CDL dataset supports composition-aware evaluation. Empirically, SLCD outperforms existing detectors on SEL, NKL, and CDL in harmony measures (HiO U), and its applicability is demonstrated in dominant vanishing point detection, reflection symmetry axis detection, and composition-based image retrieval, indicating strong practical impact for scene structure understanding.

Abstract

A novel algorithm, called semantic line combination detector (SLCD), to find an optimal combination of semantic lines is proposed in this paper. It processes all lines in each line combination at once to assess the overall harmony of the lines. First, we generate various line combinations from reliable lines. Second, we estimate the score of each line combination and determine the best one. Experimental results demonstrate that the proposed SLCD outperforms existing semantic line detectors on various datasets. Moreover, it is shown that SLCD can be applied effectively to three vision tasks of vanishing point detection, symmetry axis detection, and composition-based image retrieval. Our codes are available at https://github.com/Jinwon-Ko/SLCD.

Figures (33)

• Figure 1: After selecting reliable line candidates, there are two existing approaches to semantic line detection. The first approach in (a) focuses on locating a line near a region boundary and eliminating overlapping lines. However, a redundant line still remains, since this approach does not consider how well a group of detected lines represents the layout of a scene. The second approach in (b) takes into account only the pairwise correlation between two lines, so it may fail to assess the overall harmony of more than two semantic lines. In contrast, in (c), the proposed SLCD generates a number of line combinations, analyzes all lines in each combination at once, and then finds the most harmonious combination that conveys the global scene composition optimally.
• Figure 2: Overview of the proposed SLCD algorithm.
• Figure 3: The architecture of SLCD. CA and FC denote cross-attention and fully connected layers, respectively. The blue, red, yellow, and green boxes indicate encoding, semantic feature grouping, compositional feature extraction, and score regression, respectively.
• Figure 4: From a set of line candidates, the line detector selects $K$ reliable lines, depicted in orange. A high recall rate is achieved because a sufficient number of reliable lines are selected through NMS. Ground-truth semantic lines in green are in the insets.
• Figure 5: Visualization of semantic feature grouping results. The top row shows input images with ground-truth lines. The bottom one presents the membership maps, representing the semantic region that each pixel belongs to.