QIS : Interactive Segmentation via Quasi-Conformal Mappings

TL;DR

A thorough analysis of the proposed quasi-conformal interactive segmentation (QIS) model, which incorporates user input in the form of positive and negative clicks, is provided, including theoretical support for the ability of QIS to include or exclude regions of interest or disinterest based on the user's indication.

Abstract

Image segmentation plays a crucial role in extracting important objects of interest from images, enabling various applications. While existing methods have shown success in segmenting clean images, they often struggle to produce accurate segmentation results when dealing with degraded images, such as those containing noise or occlusions. To address this challenge, interactive segmentation has emerged as a promising approach, allowing users to provide meaningful input to guide the segmentation process. However, an important problem in interactive segmentation lies in determining how to incorporate minimal yet meaningful user guidance into the segmentation model. In this paper, we propose the quasi-conformal interactive segmentation (QIS) model, which incorporates user input in the form of positive and negative clicks. Users mark a few pixels belonging to the object region as positive clicks, indicating that the segmentation model should include a region around these clicks. Conversely, negative clicks are provided on pixels belonging to the background, instructing the model to exclude the region near these clicks from the segmentation mask. Additionally, the segmentation mask is obtained by deforming a template mask with the same topology as the object of interest using an orientation-preserving quasiconformal mapping. This approach helps to avoid topological errors in the segmentation results. We provide a thorough analysis of the proposed model, including theoretical support for the ability of QIS to include or exclude regions of interest or disinterest based on the user's indication. To evaluate the performance of QIS, we conduct experiments on synthesized images, medical images, natural images and noisy natural images. The results demonstrate the efficacy of our proposed method.

Figures (15)

• Figure 1: An illustration of the relationship between the conformality distortion and the Beltrami coefficient of a quasi-conformal mapping.
• Figure 1: A flow chart illustrating the process of the quasi-conformal interactive segmentation (QIS) model.
• Figure 1: Segmentation result achieved by employing the QIS model on a synthesized image featuring a taco on a plate, with the objective of segmenting the taco.
• Figure 2: An example demonstrating the division of isolated regions in $\mathcal{K}_i$ into multiple $D^i_j$ for $k=3$. In this example, the image domain is classified into three distinct regions based on pixel values: (1) background white, (2) blue, and (3) yellow. Subsequently, the disconnected regions within each class $\mathcal{K}_i$ are further divided into individual $D^i_j$'s.
• Figure 2: Segmentation result attained by utilizing the QIS model on a synthesized image highlighting the eating utensils, specifically the knife and fork, with the aim of segmenting the knife.

Theorems & Definitions (14)

• Definition 4.1: Click map
• Theorem 4.2
• Proof 1
• Theorem 4.3
• Theorem 4.4
• Proof 2
• Theorem 4.5
• Proof 3
• Theorem 4.6: Existence of minimizer
• Proof 4