Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Point Tracking as a Temporal Cue for Robust Myocardial Segmentation in Echocardiography Videos

Bahar Khodabakhshian, Nima Hashemi, Armin Saadat, Zahra Gholami, In-Chang Hwang, Samira Sojoudi, Christina Luong, Purang Abolmaesumi, Teresa Tsang

TL;DR

This work tackles unstable myocardium segmentation in echocardiography videos caused by noise and variability. It introduces Point-Seg, a transformer-based framework that uses a dense, learned point-tracking module trained on synthetic data as a temporal cue to guide segmentation, avoiding error-prone memory-based feature propagation. The model contains a Point Decoder and a Mask Decoder, with fusion layers and a temporal smoothing loss to ensure temporal coherence; it achieves comparable Dice to state-of-the-art methods on high-quality data and superior robustness on low-quality data, while providing pixel-level motion information for downstream tasks like strain analysis. Evaluations on CAMUS, SynUS, and a private dataset show strong performance and real-time inference (~32.8 FPS), and the code is publicly available.

Abstract

Purpose: Myocardium segmentation in echocardiography videos is a challenging task due to low contrast, noise, and anatomical variability. Traditional deep learning models either process frames independently, ignoring temporal information, or rely on memory-based feature propagation, which accumulates error over time. Methods: We propose Point-Seg, a transformer-based segmentation framework that integrates point tracking as a temporal cue to ensure stable and consistent segmentation of myocardium across frames. Our method leverages a point-tracking module trained on a synthetic echocardiography dataset to track key anatomical landmarks across video sequences. These tracked trajectories provide an explicit motion-aware signal that guides segmentation, reducing drift and eliminating the need for memory-based feature accumulation. Additionally, we incorporate a temporal smoothing loss to further enhance temporal consistency across frames. Results: We evaluate our approach on both public and private echocardiography datasets. Experimental results demonstrate that Point-Seg has statistically similar accuracy in terms of Dice to state-of-the-art segmentation models in high quality echo data, while it achieves better segmentation accuracy in lower quality echo with improved temporal stability. Furthermore, Point-Seg has the key advantage of pixel-level myocardium motion information as opposed to other segmentation methods. Such information is essential in the computation of other downstream tasks such as myocardial strain measurement and regional wall motion abnormality detection. Conclusion: Point-Seg demonstrates that point tracking can serve as an effective temporal cue for consistent video segmentation, offering a reliable and generalizable approach for myocardium segmentation in echocardiography videos. The code is available at https://github.com/DeepRCL/PointSeg.

Point Tracking as a Temporal Cue for Robust Myocardial Segmentation in Echocardiography Videos

TL;DR

This work tackles unstable myocardium segmentation in echocardiography videos caused by noise and variability. It introduces Point-Seg, a transformer-based framework that uses a dense, learned point-tracking module trained on synthetic data as a temporal cue to guide segmentation, avoiding error-prone memory-based feature propagation. The model contains a Point Decoder and a Mask Decoder, with fusion layers and a temporal smoothing loss to ensure temporal coherence; it achieves comparable Dice to state-of-the-art methods on high-quality data and superior robustness on low-quality data, while providing pixel-level motion information for downstream tasks like strain analysis. Evaluations on CAMUS, SynUS, and a private dataset show strong performance and real-time inference (~32.8 FPS), and the code is publicly available.

Abstract

Purpose: Myocardium segmentation in echocardiography videos is a challenging task due to low contrast, noise, and anatomical variability. Traditional deep learning models either process frames independently, ignoring temporal information, or rely on memory-based feature propagation, which accumulates error over time. Methods: We propose Point-Seg, a transformer-based segmentation framework that integrates point tracking as a temporal cue to ensure stable and consistent segmentation of myocardium across frames. Our method leverages a point-tracking module trained on a synthetic echocardiography dataset to track key anatomical landmarks across video sequences. These tracked trajectories provide an explicit motion-aware signal that guides segmentation, reducing drift and eliminating the need for memory-based feature accumulation. Additionally, we incorporate a temporal smoothing loss to further enhance temporal consistency across frames. Results: We evaluate our approach on both public and private echocardiography datasets. Experimental results demonstrate that Point-Seg has statistically similar accuracy in terms of Dice to state-of-the-art segmentation models in high quality echo data, while it achieves better segmentation accuracy in lower quality echo with improved temporal stability. Furthermore, Point-Seg has the key advantage of pixel-level myocardium motion information as opposed to other segmentation methods. Such information is essential in the computation of other downstream tasks such as myocardial strain measurement and regional wall motion abnormality detection. Conclusion: Point-Seg demonstrates that point tracking can serve as an effective temporal cue for consistent video segmentation, offering a reliable and generalizable approach for myocardium segmentation in echocardiography videos. The code is available at https://github.com/DeepRCL/PointSeg.
Paper Structure (15 sections, 4 equations, 2 figures, 4 tables)

This paper contains 15 sections, 4 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Point Decoder
  4. Fusion Layers and Mask Decoder
  5. Fusion Layers
  6. Mask Decoder
  7. Learning Objectives
  8. Experiments and Results
  9. Datasets
  10. Implementation Details
  11. Comparison with State-of-the-Art Methods
  12. Quantitative Results
  13. Qualitative Results
  14. Ablation Study
  15. Conclusion and Future Work

Figures (2)

  • Figure 1: Overview of the proposed model architecture, consisting of a point tracking module and a segmentation module. The point decoder tracks both manually selected and grid-based points, while the mask decoder integrates spatial and temporal cues through attention mechanisms to generate segmentation masks.
  • Figure 2: Qualitative comparison of segmentation results for one echo from CAMUS dataset. Red represents predictions, green denotes ground truth, and yellow indicates overlap. Our method shows no flickering compared to image-based models and less drift than memory-based approaches.