Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

INSTA-YOLO: Real-Time Instance Segmentation

Eslam Mohamed, Abdelrahman Shaker, Ahmad El-Sallab, Mayada Hadhoud

TL;DR

The proposed Insta-YOLO is inspired by the YOLO one-shot object detector, with the box regression loss is replaced with polynomial regression in the localization head, which enables the model to skip the segmentation up-sampling decoder altogether and produces the instance segmentation contour from the polynomial output coefficients.

Abstract

Instance segmentation has gained recently huge attention in various computer vision applications. It aims at providing different IDs to different object of the scene, even if they belong to the same class. This is useful in various scenarios, especially in occlusions. Instance segmentation is usually performed as a two-stage pipeline. First, an object is detected, then semantic segmentation within the detected box area. This process involves costly up-sampling, especially for the segmentation part. Moreover, for some applications, such as LiDAR point clouds and aerial object detection, it is often required to predict oriented boxes, which add extra complexity to the two-stage pipeline. In this paper, we propose Insta-YOLO, a novel one-stage end-to-end deep learning model for real-time instance segmentation. The proposed model is inspired by the YOLO one-shot object detector, with the box regression loss is replaced with polynomial regression in the localization head. This modification enables us to skip the segmentation up-sampling decoder altogether and produces the instance segmentation contour from the polynomial output coefficients. In addition, this architecture is a natural fit for oriented objects. We evaluate our model on three datasets, namely, Carnva, Cityscapes and Airbus. The results show our model achieves competitive accuracy in terms of mAP with significant improvement in speed by 2x on GTX-1080 GPU.

INSTA-YOLO: Real-Time Instance Segmentation

TL;DR

The proposed Insta-YOLO is inspired by the YOLO one-shot object detector, with the box regression loss is replaced with polynomial regression in the localization head, which enables the model to skip the segmentation up-sampling decoder altogether and produces the instance segmentation contour from the polynomial output coefficients.

Abstract

Instance segmentation has gained recently huge attention in various computer vision applications. It aims at providing different IDs to different object of the scene, even if they belong to the same class. This is useful in various scenarios, especially in occlusions. Instance segmentation is usually performed as a two-stage pipeline. First, an object is detected, then semantic segmentation within the detected box area. This process involves costly up-sampling, especially for the segmentation part. Moreover, for some applications, such as LiDAR point clouds and aerial object detection, it is often required to predict oriented boxes, which add extra complexity to the two-stage pipeline. In this paper, we propose Insta-YOLO, a novel one-stage end-to-end deep learning model for real-time instance segmentation. The proposed model is inspired by the YOLO one-shot object detector, with the box regression loss is replaced with polynomial regression in the localization head. This modification enables us to skip the segmentation up-sampling decoder altogether and produces the instance segmentation contour from the polynomial output coefficients. In addition, this architecture is a natural fit for oriented objects. We evaluate our model on three datasets, namely, Carnva, Cityscapes and Airbus. The results show our model achieves competitive accuracy in terms of mAP with significant improvement in speed by 2x on GTX-1080 GPU.

Paper Structure

This paper contains 18 sections, 14 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Object Detection
  4. Instance Segmentation
  5. Bounding Polygon for Instance Segmentation
  6. Methodology
  7. Data Generation
  8. Network Architecture
  9. Losses Representations
  10. Classification and Confidence loss
  11. Localization Loss
  12. Experimental Setup
  13. Datasets
  14. Training Setup
  15. Experimental Results
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Top: Fixed representation for contour points. Middle: Variable step representation. Points 2,3 show the benefit of variable step compared to fixed steps. Bottom: Another valid representation demonstrating that there is no unique set of contour points for each instance
  • Figure 2: Insta-YOLO architecture which is inspired by YOLO, the right part illustrate our adaptation in the output layer.
  • Figure 3: Top: Result on dataset using regression only. Bottom: Result using combined localization loss.
  • Figure 4: Visualization for Insta-YOLO output on Cityscapes dataset.