Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MP-PolarMask: A Faster and Finer Instance Segmentation for Concave Images

Ke-Lei Wang, Pin-Hsuan Chou, Young-Ching Chou, Chia-Jen Liu, Cheng-Kuan Lin, Yu-Chee Tseng

TL;DR

MP-PolarMask addresses the limitation of PolarMask in representing concave shapes by introducing multiple Polar systems: a main center plus four auxiliary centers, forming a five-center, multi-ray representation. The method employs a dedicated MP-Head to predict centers and rays and an MP-Assembly to fuse five ray sequences into a final mask, with losses extending PolarMask by including auxiliary-centric terms. Empirically, MP-PolarMask yields notable gains on COCO, particularly for concave food objects (e.g., AP_L and AP improvements of 13.69% and 7.23% respectively on the food subset), while maintaining competitive speed. This work demonstrates that a multi-point polar representation can deliver finer instance segmentation suitable for real-time applications, and points to future improvements in auxiliary-point flexibility and mask assembly strategies.

Abstract

While there are a lot of models for instance segmentation, PolarMask stands out as a unique one that represents an object by a Polar coordinate system. With an anchor-box-free design and a single-stage framework that conducts detection and segmentation at one time, PolarMask is proved to be able to balance efficiency and accuracy. Hence, it can be easily connected with other downstream real-time applications. In this work, we observe that there are two deficiencies associated with PolarMask: (i) inability of representing concave objects and (ii) inefficiency in using ray regression. We propose MP-PolarMask (Multi-Point PolarMask) by taking advantage of multiple Polar systems. The main idea is to extend from one main Polar system to four auxiliary Polar systems, thus capable of representing more complicated convex-and-concave-mixed shapes. We validate MP-PolarMask on both general objects and food objects of the COCO dataset, and the results demonstrate significant improvement of 13.69% in AP_L and 7.23% in AP over PolarMask with 36 rays.

MP-PolarMask: A Faster and Finer Instance Segmentation for Concave Images

TL;DR

MP-PolarMask addresses the limitation of PolarMask in representing concave shapes by introducing multiple Polar systems: a main center plus four auxiliary centers, forming a five-center, multi-ray representation. The method employs a dedicated MP-Head to predict centers and rays and an MP-Assembly to fuse five ray sequences into a final mask, with losses extending PolarMask by including auxiliary-centric terms. Empirically, MP-PolarMask yields notable gains on COCO, particularly for concave food objects (e.g., AP_L and AP improvements of 13.69% and 7.23% respectively on the food subset), while maintaining competitive speed. This work demonstrates that a multi-point polar representation can deliver finer instance segmentation suitable for real-time applications, and points to future improvements in auxiliary-point flexibility and mask assembly strategies.

Abstract

While there are a lot of models for instance segmentation, PolarMask stands out as a unique one that represents an object by a Polar coordinate system. With an anchor-box-free design and a single-stage framework that conducts detection and segmentation at one time, PolarMask is proved to be able to balance efficiency and accuracy. Hence, it can be easily connected with other downstream real-time applications. In this work, we observe that there are two deficiencies associated with PolarMask: (i) inability of representing concave objects and (ii) inefficiency in using ray regression. We propose MP-PolarMask (Multi-Point PolarMask) by taking advantage of multiple Polar systems. The main idea is to extend from one main Polar system to four auxiliary Polar systems, thus capable of representing more complicated convex-and-concave-mixed shapes. We validate MP-PolarMask on both general objects and food objects of the COCO dataset, and the results demonstrate significant improvement of 13.69% in AP_L and 7.23% in AP over PolarMask with 36 rays.
Paper Structure (13 sections, 15 equations, 7 figures, 2 tables, 1 algorithm)

This paper contains 13 sections, 15 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Anchor-based and Anchor-free Detection
  4. Instance Segmentation
  5. PolarMask
  6. MP-PolarMask
  7. MP-Head
  8. MP-Assembly
  9. Loss Functions
  10. Experiment Results
  11. Validation on General Objects
  12. Validation on Food Objects
  13. Conclusions

Figures (7)

  • Figure 1: Comparison of Polarmask and MP-PolarMask.
  • Figure 2: PolarMask 9157078.
  • Figure 3: Left: PolarMask; Right: MP-PolarMask
  • Figure 4: The architecture of MP-PolarMask.
  • Figure 5: A running example of MP-PolarMask: (a) the input image, (b) the main center and four auxiliary centers, (c) the mask points expanded from the main center, (d) the sequence $X'_4$ in Quadrant $4$ refined by the angle $\alpha_4$, (e) ($a_m, b_m$) and its corresponding $X'_m$, where the yellow regions are to be filled by the mask points of the main center, and (f) the final mask $X'_1 | X'_2 | X_0^{Q_2 \rightarrow Q_3} | X'_3 | X_0^{Q_3 \rightarrow Q_4} | X'_4$.
  • ...and 2 more figures