Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SPIRIT: Adapting Vision Foundation Models for Unified Single- and Multi-Frame Infrared Small Target Detection

Qian Xu, Xi Li, Fei Gao, Jie Guo, Haojuan Yuan, Shuaipeng Fan, Mingjin Zhang

TL;DR

SPIRIT is proposed, a unified and VFM-compatible framework that adapts VFMs to IRSTD via lightweight physics-informed plug-ins and refines features by approximating rank-sparsity decomposition to suppress structured background components and enhance sparse target-like signals.

Abstract

Infrared small target detection (IRSTD) is crucial for surveillance and early-warning, with deployments spanning both single-frame analysis and video-mode tracking. A practical solution should leverage vision foundation models (VFMs) to mitigate infrared data scarcity, while adopting a memory-attention-based temporal propagation framework that unifies single- and multi-frame inference. However, infrared small targets exhibit weak radiometric signals and limited semantic cues, which differ markedly from visible-spectrum imagery. This modality gap makes direct use of semantics-oriented VFMs and appearance-driven cross-frame association unreliable for IRSTD: hierarchical feature aggregation can submerge localized target peaks, and appearance-only memory attention becomes ambiguous, leading to spurious clutter associations. To address these challenges, we propose SPIRIT, a unified and VFM-compatible framework that adapts VFMs to IRSTD via lightweight physics-informed plug-ins. Spatially, PIFR refines features by approximating rank-sparsity decomposition to suppress structured background components and enhance sparse target-like signals. Temporally, PGMA injects history-derived soft spatial priors into memory cross-attention to constrain cross-frame association, enabling robust video detection while naturally reverting to single-frame inference when temporal context is absent. Experiments on multiple IRSTD benchmarks show consistent gains over VFM-based baselines and SOTA performance.

SPIRIT: Adapting Vision Foundation Models for Unified Single- and Multi-Frame Infrared Small Target Detection

TL;DR

SPIRIT is proposed, a unified and VFM-compatible framework that adapts VFMs to IRSTD via lightweight physics-informed plug-ins and refines features by approximating rank-sparsity decomposition to suppress structured background components and enhance sparse target-like signals.

Abstract

Infrared small target detection (IRSTD) is crucial for surveillance and early-warning, with deployments spanning both single-frame analysis and video-mode tracking. A practical solution should leverage vision foundation models (VFMs) to mitigate infrared data scarcity, while adopting a memory-attention-based temporal propagation framework that unifies single- and multi-frame inference. However, infrared small targets exhibit weak radiometric signals and limited semantic cues, which differ markedly from visible-spectrum imagery. This modality gap makes direct use of semantics-oriented VFMs and appearance-driven cross-frame association unreliable for IRSTD: hierarchical feature aggregation can submerge localized target peaks, and appearance-only memory attention becomes ambiguous, leading to spurious clutter associations. To address these challenges, we propose SPIRIT, a unified and VFM-compatible framework that adapts VFMs to IRSTD via lightweight physics-informed plug-ins. Spatially, PIFR refines features by approximating rank-sparsity decomposition to suppress structured background components and enhance sparse target-like signals. Temporally, PGMA injects history-derived soft spatial priors into memory cross-attention to constrain cross-frame association, enabling robust video detection while naturally reverting to single-frame inference when temporal context is absent. Experiments on multiple IRSTD benchmarks show consistent gains over VFM-based baselines and SOTA performance.
Paper Structure (18 sections, 13 equations, 8 figures, 3 tables)

This paper contains 18 sections, 13 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Single-frame IR Small Target Detection.
  4. Video IRSTD and Temporal Association.
  5. Vision Foundation Models and Adaptation.
  6. Method
  7. Overview.
  8. Physics-Informed Feature Refinement
  9. Prior-Guided Memory Attention
  10. Experiments
  11. Datasets and Evaluation Metrics
  12. Datasets.
  13. Evaluation Metrics
  14. Implementation Details
  15. Main Results
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Performance comparison on multi-frame (left) and single-frame (right) IRSTD datasets. All metrics are normalized for clear visualization. Larger enclosed areas indicate superior performance.
  • Figure 2: The overall architecture of the proposed method. It integrates PIFR into the VFM backbone to decouple background and target for feature enhancement, and employs prior guided memory attention to suppress spurious associations by leveraging a feasibility field to focus on physically plausible regions.
  • Figure 3: Illustration of the PIFR module, which utilizes ridge projection and shrinkage for feature refinement.
  • Figure 4: The workflow of PGMA. By utilizing a Gaussian feasibility field as a prior for gated encoding and bias injection, this mechanism guides the model to focus on physically plausible regions to suppress spurious associations.
  • Figure 5: Visual comparison of results from different methods on the IRSTD-1k, IRDST, and IRSTD-15k datasets. Boxes in green and red represent ground-truth and detected targets, respectively.
  • ...and 3 more figures