Beyond Full Labels: Energy-Double-Guided Single-Point Prompt for Infrared Small Target Label Generation

TL;DR

A learning-based single-point prompt paradigm for infrared small target label generation (IRSTLG) to lobber annotation burdens is pioneer, aiming to adeptly transform a coarse IRSTD network into a refined label generation method.

Abstract

We pioneer a learning-based single-point prompt paradigm for infrared small target label generation (IRSTLG) to lobber annotation burdens. Unlike previous clustering-based methods, our intuition is that point-guided mask generation just requires one more prompt than target detection, i.e., IRSTLG can be treated as an infrared small target detection (IRSTD) with the location hint. Therefore, we propose an elegant yet effective Energy-Double-Guided Single-point Prompt (EDGSP) framework, aiming to adeptly transform a coarse IRSTD network into a refined label generation method. Specifically, EDGSP comprises three key modules: 1) target energy initialization (TEI), which establishes a foundational outline to streamline the mapping process for effective shape evolution, 2) double prompt embedding (DPE) for rapidly localizing interesting regions and reinforcing high-resolution individual edges to avoid label adhesion, and 3) bounding box-based matching (BBM) for eliminating false masks via considering comprehensive cluster boundary conditions to obtain a reliable output. In this way, pseudo labels generated by three backbones equipped with our EDGSP achieve 100% object-level probability of detection (Pd) and 0% false-alarm rate (Fa) on SIRST, NUDT-SIRST, and IRSTD-1k datasets, with a pixel-level intersection over union (IoU) improvement of 13.28% over state-of-the-art (SOTA) label generation methods. Further applying our inferred masks to train detection models, EDGSP, for the first time, enables a single-point-generated pseudo mask to surpass the manual labels. Even with coarse single-point annotations, it still achieves 99.5% performance of full labeling. Code is available at https://github.com/xdFai/EDGSP.

Figures (9)

• Figure 1: Comparing the result of target detection and label generation on mixed datasets from SIRST, NUDT-SIRST, and IRSTD-1k in $IoU(\%)$, ${{P}_{d}}(\%)$, and ${{F}_{a}}({10}^{-6})$. EDGSP-Coa. denotes EDGSP with the coarse single-point prompt.
• Figure 2: (a) The proposed target energy initialization not only simplifies the shape evolution process but provides a more sufficient target outline representation. (b) Prompts equipped with TEI enhance grayscale and gradient differences between neighboring targets, motivating us to embed the prompt at the model's end to prevent label adhesion. (c) Bounding box-based matching showcases more stable false alarm removal capability than eight-connective regions matching (ERM) Ying.
• Figure 3: Illustration of the proposed energy-double-guided single-point prompt (EDGSP) for infrared small target label generation. It consists of three parts. (a) Target energy initialization (TEI) for sufficient shape evolution, (b) double prompt embedding (DPE) to reinforce individual differences and prevent mask adhesion, and (c) bounding box-based matching (BBM) to eliminate false annotations.
• Figure 4: Visualization and correction of seven errors in the NUDT-SIRST DNA-Net and IRSTD-1k Zhang_IS datasets. Error annotations are highlighted in yellow circles.
• Figure 5: Visual results of seven IRSTLG methods on the SIRST, NUDT-SIRST, IRSTD-1k. EDGSP-Coa. represents the pseudo label of EDGSP using the coarse single-point prompt. Small flaws in the ground truth are highlighted with arrows. For clarity, only annotated areas are shown.