FeedbackSTS-Det: Sparse Frames-Based Spatio-Temporal Semantic Feedback Network for Infrared Small Target Detection

TL;DR

The paper tackles infrared small target detection in challenging cluttered scenes by introducing FeedbackSTS-Det, a sparse-frames based spatio-temporal semantic feedback network. It uses a closed-loop framework with forward and backward refinement modules integrated into a 3D-ResUNet backbone, augmented by a Sparse Semantic Module to capture long-range temporal dependencies with low compute. The approach demonstrates state-of-the-art performance on public ISTD datasets, with comprehensive ablations validating the contributions of the spatio-temporal feedback strategy, the SSM, and the 3D architecture. The work offers practical benefits in robust false alarm suppression and consistent training-to-inference transfer, though it acknowledges evaluation over a broader set of scenarios as future work.

Abstract

Infrared small target detection (ISTD) under complex backgrounds remains a critical yet challenging task, primarily due to the extremely low signal-to-clutter ratio, persistent dynamic interference, and the lack of distinct target features. While multi-frame detection methods leverages temporal cues to improve upon single-frame approaches, existing methods still struggle with inefficient long-range dependency modeling and insufficient robustness. To overcome these issues, we propose a novel scheme for ISTD, realized through a sparse frames-based spatio-temporal semantic feedback network named FeedbackSTS-Det. The core of our approach is a novel spatio-temporal semantic feedback strategy with a closed-loop semantic association mechanism, which consists of paired forward and backward refinement modules that work cooperatively across the encoder and decoder. Moreover, both modules incorporate an embedded sparse semantic module (SSM), which performs structured sparse temporal modeling to capture long-range dependencies with low computational cost. This integrated design facilitates robust implicit inter-frame registration and continuous semantic refinement, effectively suppressing false alarms. Furthermore, our overall procedure maintains a consistent training-inference pipeline, which ensures reliable performance transfer and increases model robustness. Extensive experiments on multiple benchmark datasets confirm the effectiveness of FeedbackSTS-Det. Code and models are available at: https://github.com/IDIP-Lab/FeedbackSTS-Det.

Figures (15)

• Figure 1: Overall procedure of FeedbackSTS-Det model for ISTD. (a) presents the overall framework. (b) denotes forward spatio-temporal semantic refinement module (FSTSRM), (c) illustrates backward spatio-temporal semantic refinement module (BSTSRM), and (d) describes sparse semantic module (SSM).
• Figure 2: Basic feedback module (BFBM). (a) presents the framework of the BFBM module, which is embedded within the SSM. (b) details the procedure of feature extraction in BFBM, and (c) shows the procedure of feature alignment module in BFBM.
• Figure 3: ROC curves on two benchmark datasets. (a) ROC Curve on NUDT-MIRSDT. (b) ROC Curve on IRSTVideo-LEO.
• Figure 4: Qualitative results of different methods. For better visual presentation, the target regions are highlighted with red boxes and then displayed as zoomed-in views. Missed detections and false alarms are marked with blue and yellow circles, respectively.
• Figure 5: Comparison of different feedback design variants against the Full-FB. Evaluation metrics include mean Intersection over $mIoU$, $F_1$, $P_d$, $FSR$, and $AUC$, where $\Delta$ represents the performance difference relative to the Full-FB.(a) Dec-NoFB vs Full-FB. (b) Enc-NoFB vs Full-FB. (c) All-Fwd vs Full-FB. (d) All-Bwd vs Full-FB. (e) Part-FB1 vs Full-FB. (f) Part-FB2 vs Full-FB.