Ada-Tracker: Soft Tissue Tracking via Inter-Frame and Adaptive-Template Matching

TL;DR

Ada-Tracker addresses soft tissue tracking in surgical scenes by unifying patch-level optical flow with adaptive-template updating. The method comprises inter-frame matching to capture short-term tissue motion via a RAFT-based flow with confidence and occlusion cues, plus adaptive-template matching using scale-invariant warping and an anchor-based matcher for robust, long-term tracking. Training uses cycle-consistency, photometric, augmentation, and smoothness losses to self-supervise both stages. On the SurgT benchmark, Ada-Tracker achieves superior accuracy and robustness against state-of-the-art trackers, demonstrating strong potential for real-time intervention support with deformations, occlusions, and appearance changes.

Abstract

Soft tissue tracking is crucial for computer-assisted interventions. Existing approaches mainly rely on extracting discriminative features from the template and videos to recover corresponding matches. However, it is difficult to adopt these techniques in surgical scenes, where tissues are changing in shape and appearance throughout the surgery. To address this problem, we exploit optical flow to naturally capture the pixel-wise tissue deformations and adaptively correct the tracked template. Specifically, we first implement an inter-frame matching mechanism to extract a coarse region of interest based on optical flow from consecutive frames. To accommodate appearance change and alleviate drift, we then propose an adaptive-template matching method, which updates the tracked template based on the reliability of the estimates. Our approach, Ada-Tracker, enjoys both short-term dynamics modeling by capturing local deformations and long-term dynamics modeling by introducing global temporal compensation. We evaluate our approach on the public SurgT benchmark, which is generated from Hamlyn, SCARED, and Kidney boundary datasets. The experimental results show that Ada-Tracker achieves superior accuracy and performs more robustly against prior works. Code is available at https://github.com/wrld/Ada-Tracker.

Figures (4)

• Figure 1: Different approaches to employ optical flow for soft tissue tracking, with blue and green bounding box indicating the predicted result and ground truth. a) Inter-frame matching updates the target box based on optical flow from consecutive frames, prone to drift due to accumulated error. b) Template-based matching predicts the target box by matching the template image from the anchor box with every input frame, struggling with appearance variation and large motion. c) Our Ada-Tracker exploits the benefits of a) and b) for the more robust and accurate tracker.
• Figure 2: Overview of our proposed Ada-Tracker. Given a bounding box at the start of the surgical video, we aim to locate the ROI of soft tissues in the form of bounding boxes at every frame. Our method consists of two stages: inter-frame matching and adaptive-template matching. In the first stage, we capture the immediate motion, and obtain a coarse bounding box prediction. Next, we update the template adaptively based on the flow, confidence and occlusion information from the previous stage. We finally match the updated template and coarse ROI to obtain the final prediction.
• Figure 3: Qualitative results. We visualize the tracking in different cases, including tissue deformation, camera movement, and illumination variations.
• Figure 4: Qualitative comparison of different approaches in both short-term (first row) and long-term (second row) tracking case.