Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

You Are Your Best Teacher: Semi-Supervised Surgical Point Tracking with Cycle-Consistent Self-Distillation

Valay Bundele, Mehran Hosseinzadeh, Hendrik Lensch

TL;DR

The paper addresses the challenge of transferring synthetic-trained point trackers to surgical videos under severe domain shift and scarce annotations. It introduces SurgTracker, a semi-supervised framework that uses a single, architecture-aligned fixed teacher to generate pseudo-labels and applies cycle-consistent filtering to ensure temporal coherence, enabling stable supervision for the student. Experiments on the STIR benchmark with 80 unlabeled videos demonstrate that SurgTracker outperforms strong baselines, and ablations show that cycle consistency and single-teacher supervision are crucial for robustness in high-shift domains. This approach enables robust, real-time surgical point tracking in data-scarce settings, reducing the need for extensive labeled data and heavy teacher ensembles.

Abstract

Synthetic datasets have enabled significant progress in point tracking by providing large-scale, densely annotated supervision. However, deploying these models in real-world domains remains challenging due to domain shift and lack of labeled data-issues that are especially severe in surgical videos, where scenes exhibit complex tissue deformation, occlusion, and lighting variation. While recent approaches adapt synthetic-trained trackers to natural videos using teacher ensembles or augmentation-heavy pseudo-labeling pipelines, their effectiveness in high-shift domains like surgery remains unexplored. This work presents SurgTracker, a semi-supervised framework for adapting synthetic-trained point trackers to surgical video using filtered self-distillation. Pseudo-labels are generated online by a fixed teacher-identical in architecture and initialization to the student-and are filtered using a cycle consistency constraint to discard temporally inconsistent trajectories. This simple yet effective design enforces geometric consistency and provides stable supervision throughout training, without the computational overhead of maintaining multiple teachers. Experiments on the STIR benchmark show that SurgTracker improves tracking performance using only 80 unlabeled videos, demonstrating its potential for robust adaptation in high-shift, data-scarce domains.

You Are Your Best Teacher: Semi-Supervised Surgical Point Tracking with Cycle-Consistent Self-Distillation

TL;DR

The paper addresses the challenge of transferring synthetic-trained point trackers to surgical videos under severe domain shift and scarce annotations. It introduces SurgTracker, a semi-supervised framework that uses a single, architecture-aligned fixed teacher to generate pseudo-labels and applies cycle-consistent filtering to ensure temporal coherence, enabling stable supervision for the student. Experiments on the STIR benchmark with 80 unlabeled videos demonstrate that SurgTracker outperforms strong baselines, and ablations show that cycle consistency and single-teacher supervision are crucial for robustness in high-shift domains. This approach enables robust, real-time surgical point tracking in data-scarce settings, reducing the need for extensive labeled data and heavy teacher ensembles.

Abstract

Synthetic datasets have enabled significant progress in point tracking by providing large-scale, densely annotated supervision. However, deploying these models in real-world domains remains challenging due to domain shift and lack of labeled data-issues that are especially severe in surgical videos, where scenes exhibit complex tissue deformation, occlusion, and lighting variation. While recent approaches adapt synthetic-trained trackers to natural videos using teacher ensembles or augmentation-heavy pseudo-labeling pipelines, their effectiveness in high-shift domains like surgery remains unexplored. This work presents SurgTracker, a semi-supervised framework for adapting synthetic-trained point trackers to surgical video using filtered self-distillation. Pseudo-labels are generated online by a fixed teacher-identical in architecture and initialization to the student-and are filtered using a cycle consistency constraint to discard temporally inconsistent trajectories. This simple yet effective design enforces geometric consistency and provides stable supervision throughout training, without the computational overhead of maintaining multiple teachers. Experiments on the STIR benchmark show that SurgTracker improves tracking performance using only 80 unlabeled videos, demonstrating its potential for robust adaptation in high-shift, data-scarce domains.
Paper Structure (17 sections, 2 equations, 2 figures, 3 tables)

This paper contains 17 sections, 2 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Point Tracking
  4. Unsupervised Domain Adaptation
  5. Method
  6. Problem Formulation
  7. SurgTracker
  8. Pseudo-Label Generation
  9. Cycle-Consistent Filtering
  10. Student Fine-Tuning
  11. Experiments
  12. Datasets and Metrics
  13. Implementation Details
  14. Results
  15. Ablation Studies
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overview of the SurgTracker framework. Given an unlabeled surgical video, pseudo-labels are generated by a frozen teacher network and filtered using a cycle consistency check to remove temporally inconsistent trajectories. The filtered trajectories supervise the student model, which is fine-tuned using a tracking loss $\mathcal{L}_{\text{track}}$. The teacher model remains frozen during training.
  • Figure 2: Comparison of CoTracker3 and our method on a challenging sequence. Red and green dots mark initial and mid-frame predicted positions, respectively, blue lines show trajectories, and pink lines indicate final error. Our model better handles occlusion and motion change, accurately recovering the original trajectory.