SPAN: Continuous Modeling of Suspicion Progression for Temporal Intention Localization

TL;DR

This work targets Temporal Intention Localization (TIL) in video surveillance, where suspicion evolves continuously rather than jumping between discrete levels. SPAN reframes TIL as continuous suspicion progression using a Temporal Point Process–inspired formulation, introducing a kernel-based suspicion score s(t) that accumulates the effects of current and past actions through duration and frequency terms, $D(i)$ and $N(i)$, and a dynamic decay $\gamma$. A Multimodal Modulator with a Concept-Anchored Mapping connects observed actions to predefined intention concepts, enabling adaptive coefficient modulation for $\gamma$, $\alpha$, and $\beta$ based on visual, confidence, and temporal cues. Experiments on the HAI dataset show substantial improvements over discrete baselines, including a $19.8\%$ reduction in MSE and a $1.78\%$ rise in avg $mAP$, with notable gains in low-frequency scenarios, illustrating the method’s capacity for early, interpretable, context-aware intervention. The approach also demonstrates robustness across indoor/outdoor environments and provides enhanced explainability through action-to-intention mappings and continuous suspicion trajectories.

Abstract

Temporal Intention Localization (TIL) is crucial for video surveillance, focusing on identifying varying levels of suspicious intentions to improve security monitoring. However, existing discrete classification methods fail to capture the continuous nature of suspicious intentions, limiting early intervention and explainability. In this paper, we propose the Suspicion Progression Analysis Network (SPAN), which shifts from discrete classification to continuous regression, enabling the capture of fluctuating and evolving suspicious intentions. We reveal that suspicion exhibits long-term dependencies and cumulative effects, similar to Temporal Point Process (TPP) theory. Based on these insights, we define a suspicion score formula that models continuous changes while accounting for temporal characteristics. We also introduce Suspicion Coefficient Modulation, which adjusts suspicion coefficients using multimodal information to reflect the varying impacts of suspicious actions. Additionally, the Concept-Anchored Mapping method is proposed to link suspicious actions to predefined intention concepts, offering insights into both the actions and their potential underlying intentions. Extensive experiments on the HAI dataset show that SPAN significantly outperforms existing methods, reducing MSE by 19.8% and improving average mAP by 1.78%. Notably, SPAN achieves a 2.74% mAP gain in low-frequency cases, demonstrating its superior ability to capture subtle behavioral changes. Compared to discrete classification systems, our continuous suspicion modeling approach enables earlier detection and proactive intervention, greatly enhancing system explainability and practical utility in security applications.

Figures (6)

• Figure 1: Comparison between discrete and our continuous SPAN. The discrete method classifies suspicion into three fixed levels via visual features, while SPAN models suspicion as a continuous progression through multimodal features, enabling more precise, earlier detection of suspicious intentions with improved explainability
• Figure 2: Motivation for our approach: (a) Temporal dependencies and cumulative effects in suspicion scores, (b) Comparison of textual and visual feature similarities.
• Figure 3: Overview of our Suspicion Progression Analysis Network (SPAN). The framework consists of: (1) TPP Suspicion Modeling, which formalizes suspicion scores capturing continuous changes and temporal dependencies; and (2) Suspicion Coefficient Modulation with our Concept-Anchored Mapping method that associates suspicious actions with underlying intention concepts. Together, these components effectively model the continuous progression of hidden intentions.
• Figure 4: Performance comparison on different HAI scenario subsets (table) and their visualization (graph). The best scores in each column are highlighted in bold. Results of our method are shaded in gray.
• Figure 5: Visualization of suspicion scores with baseline and different modulation methods.