Exploring Temporally-Aware Features for Point Tracking

TL;DR

This paper addresses robust point tracking in videos by shifting focus from heavy refinement to a temporally aware backbone. It introduces Chrono, a temporally aware feature backbone that fuses pre-trained DINOv2 representations with a temporal adapter to capture long-range temporal context, enabling accurate, refiner-free point prediction via simple feature matching. Chrono achieves state-of-the-art performance on TAP-Vid datasets with high efficiency, and remains compatible with optional refiners for additional gains. The approach has practical implications for real-time video understanding in robotics, editing, and related applications due to its strong accuracy and reduced computational burden.

Abstract

Point tracking in videos is a fundamental task with applications in robotics, video editing, and more. While many vision tasks benefit from pre-trained feature backbones to improve generalizability, point tracking has primarily relied on simpler backbones trained from scratch on synthetic data, which may limit robustness in real-world scenarios. Additionally, point tracking requires temporal awareness to ensure coherence across frames, but using temporally-aware features is still underexplored. Most current methods often employ a two-stage process: an initial coarse prediction followed by a refinement stage to inject temporal information and correct errors from the coarse stage. These approach, however, is computationally expensive and potentially redundant if the feature backbone itself captures sufficient temporal information. In this work, we introduce Chrono, a feature backbone specifically designed for point tracking with built-in temporal awareness. Leveraging pre-trained representations from self-supervised learner DINOv2 and enhanced with a temporal adapter, Chrono effectively captures long-term temporal context, enabling precise prediction even without the refinement stage. Experimental results demonstrate that Chrono achieves state-of-the-art performance in a refiner-free setting on the TAP-Vid-DAVIS and TAP-Vid-Kinetics datasets, among common feature backbones used in point tracking as well as DINOv2, with exceptional efficiency. Project page: https://cvlab-kaist.github.io/Chrono/

Figures (6)

• Figure 1: Chrono is a highly precise, temporally-aware feature backbone specifically designed for point tracking. Unlike prior methods that rely on heavy iterative refinement for each query point harley2022particledoersch2022tapdoersch2023tapircho2024localkaraev2023cotracker or test-time optimization tumanyan2025dinowang2023tracking, Chrono achieves competitive tracking performance through simple feature matching using a non-parametric operation. Moreover, its performance can be further improved by an optional refinement step.
• Figure 2: Comparison of initial trajectories from Chrono and existing feature backbone for point tracking. Chrono demonstrates a significant improvement over existing feature backbones such as TSM-ResNet lin2019tsm.
• Figure 3: Overall architecture of Chrono. Temporal adapters between transformer blocks use 2D convolution and 1D local attention along the temporal axis to output temporally-aware features.
• Figure 4: Point track prediction. To predict point positions, we simply match the query points with features from other frames, without using any learnable layers.
• Figure 5: Qualitative comparison of complex real-world video tracking. We qualitatively compare the results generated by Chrono with those from other commonly used backbones in point tracking and as well as DINOv2. Our model demonstrates better smoothness and precision than other competitors.