DSTED: Decoupling Temporal Stabilization and Discriminative Enhancement for Surgical Workflow Recognition

TL;DR

The paper tackles temporal jitter and ambiguous phase discrimination in surgical workflow recognition. It introduces DSTED, a dual-pathway framework that decouples temporal stabilization (Reliable Memory Propagation) from discriminative enhancement (Uncertainty-Aware Prototype Retrieval) and combines them via a confidence-driven gate. Empirical results on AutoLaparo show state-of-the-art accuracy and F1, with ablations confirming complementary gains and reduced temporal jitter. The work provides a robust, clinically relevant approach for context-aware assistance in complex surgical procedures.

Abstract

Purpose: Surgical workflow recognition enables context-aware assistance and skill assessment in computer-assisted interventions. Despite recent advances, current methods suffer from two critical challenges: prediction jitter across consecutive frames and poor discrimination of ambiguous phases. This paper aims to develop a stable framework by selectively propagating reliable historical information and explicitly modeling uncertainty for hard sample enhancement. Methods: We propose a dual-pathway framework DSTED with Reliable Memory Propagation (RMP) and Uncertainty-Aware Prototype Retrieval (UPR). RMP maintains temporal coherence by filtering and fusing high-confidence historical features through multi-criteria reliability assessment. UPR constructs learnable class-specific prototypes from high-uncertainty samples and performs adaptive prototype matching to refine ambiguous frame representations. Finally, a confidence-driven gate dynamically balances both pathways based on prediction certainty. Results: Our method achieves state-of-the-art performance on AutoLaparo-hysterectomy with 84.36% accuracy and 65.51% F1-score, surpassing the second-best method by 3.51% and 4.88% respectively. Ablations reveal complementary gains from RMP (2.19%) and UPR (1.93%), with synergistic effects when combined. Extensive analysis confirms substantial reduction in temporal jitter and marked improvement on challenging phase transitions. Conclusion: Our dual-pathway design introduces a novel paradigm for stable workflow recognition, demonstrating that decoupling the modeling of temporal consistency and phase ambiguity yields superior performance and clinical applicability.

Figures (6)

• Figure 1: (a) Prediction jitter across adjacent frames, where phase classifications fluctuate significantly despite minimal visual changes. (b) Ambiguous phase predictions caused by class imbalance and high inter-phase similarity, leading to confusion between visually overlapping surgical stages.
• Figure 2: Overview of the proposed framework with reliable memory propagation and uncertainty-aware prototype retrieval, integrated via confidence-driven gating.
• Figure 3: Confusion matrices on the AutoLaparo dataset comparing TGMA with all baseline methods.
• Figure 4: Visualization of temporal prediction continuity, comparing the baseline and the model with the proposed RMP module across consecutive frames.
• Figure 5: Visualization of prediction improvements on ambiguous phases, illustrating the performance of the baseline, the model with UPR only, and the complete DSTED framework.