Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interactive Instance Annotation with Siamese Networks

Xiang Xu, Ruotong Li, Mengjun Yi, Baile XU, Furao Shen, Jian Zhao

TL;DR

SiamAnno is the first model to explore Siamese architecture for instance annotation and achieves state-of-the-art (SOTA) performance across multiple datasets, demonstrating its ability to handle domain and environment shifts in cross-domain tasks.

Abstract

Annotating instance masks is time-consuming and labor-intensive. A promising solution is to predict contours using a deep learning model and then allow users to refine them. However, most existing methods focus on in-domain scenarios, limiting their effectiveness for cross-domain annotation tasks. In this paper, we propose SiamAnno, a framework inspired by the use of Siamese networks in object tracking. SiamAnno leverages one-shot learning to annotate previously unseen objects by taking a bounding box as input and predicting object boundaries, which can then be adjusted by annotators. Trained on one dataset and tested on another without fine-tuning, SiamAnno achieves state-of-the-art (SOTA) performance across multiple datasets, demonstrating its ability to handle domain and environment shifts in cross-domain tasks. We also provide more comprehensive results compared to previous work, establishing a strong baseline for future research. To our knowledge, SiamAnno is the first model to explore Siamese architecture for instance annotation.

Interactive Instance Annotation with Siamese Networks

TL;DR

SiamAnno is the first model to explore Siamese architecture for instance annotation and achieves state-of-the-art (SOTA) performance across multiple datasets, demonstrating its ability to handle domain and environment shifts in cross-domain tasks.

Abstract

Annotating instance masks is time-consuming and labor-intensive. A promising solution is to predict contours using a deep learning model and then allow users to refine them. However, most existing methods focus on in-domain scenarios, limiting their effectiveness for cross-domain annotation tasks. In this paper, we propose SiamAnno, a framework inspired by the use of Siamese networks in object tracking. SiamAnno leverages one-shot learning to annotate previously unseen objects by taking a bounding box as input and predicting object boundaries, which can then be adjusted by annotators. Trained on one dataset and tested on another without fine-tuning, SiamAnno achieves state-of-the-art (SOTA) performance across multiple datasets, demonstrating its ability to handle domain and environment shifts in cross-domain tasks. We also provide more comprehensive results compared to previous work, establishing a strong baseline for future research. To our knowledge, SiamAnno is the first model to explore Siamese architecture for instance annotation.
Paper Structure (15 sections, 4 equations, 5 figures, 4 tables)

This paper contains 15 sections, 4 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Siamese Networks
  4. Instance Annotation
  5. Proposed Methods
  6. Overview
  7. Network Architecture
  8. Contour Prediction Head
  9. Loss Function
  10. Experiments
  11. Evaluation Metric
  12. Comparisons with State-of-the-Arts
  13. In-Domain Annotation
  14. Cross-Domain Annotation
  15. Conclusion

Figures (5)

  • Figure 1: SiamAnno for instance annotation. Annotators wrap the instance by dragging a bounding box or input the bounding box predicted by some object detection model. SiamAnno takes the the bounded region with a certain slack as the input of its two branches, and outputs the predicted contour. Users can further correct the estimated boundary by pulling the vertices.
  • Figure 2: The network architecture of SiamAnno. Features from the search branch and the target branch are correlated to produce a fused feature map. We expand the feature map by combining low-level features and send it to the contour prediction head.
  • Figure 3: In-domain annotation results on Cityscapes. Compared to the car, inaccurate contours usually happen to the person who may have irregular shape or movement (the leftmost image).
  • Figure 4: Cross-domain annotation results on KITTI (the first column), ADE20k (the second column) and Rooftop (the third column). Note that the model here are only trained on Cityscapes without training or finetuning on these datasets.
  • Figure 5: Comparison between the per-component mode (left) and the per-instance mode (right). Due to its nature of boundary shrinking, SiamAnno is not good at regressing object boundaries with breaks.