The Temporal Trap: Entanglement in Pre-Trained Visual Representations for Visuomotor Policy Learning

TL;DR

This work reveals temporal entanglement as a core limitation when leveraging time-invariant pre-trained visual representations for visuomotor policy learning. It introduces robust probes to quantify short-range and long-range temporal entanglement and demonstrates strong correlations with policy performance. A simple temporal disentanglement baseline, based on embedding the task progression signal, outperforms conventional feature-augmentation methods and enables substantial policy gains. The findings argue for temporally aware PVRs and provide practical baselines and insights that can guide future development of temporally structured representations for robotic control.

Abstract

The integration of pre-trained visual representations (PVRs) has significantly advanced visuomotor policy learning. However, effectively leveraging these models remains a challenge. We identify temporal entanglement as a critical, inherent issue when using these time-invariant models in sequential decision-making tasks. This entanglement arises because PVRs, optimised for static image understanding, struggle to represent the temporal dependencies crucial for visuomotor control. In this work, we quantify the impact of temporal entanglement, demonstrating a strong correlation between a policy's success rate and the ability of its latent space to capture task-progression cues. Based on these insights, we propose a simple, yet effective disentanglement baseline designed to mitigate temporal entanglement. Our empirical results show that traditional methods aimed at enriching features with temporal components are insufficient on their own, highlighting the necessity of explicitly addressing temporal disentanglement for robust visuomotor policy learning.

Figures (7)

• Figure 1: PCA of features from an expert demonstration in Bin Picking across PVRs (Top row: ViT models; Bottom row: ResNet models). Frame colours align with trajectory stages, suggesting feature entanglement during the gripper descent and ascent, and during the gripper stop phase. Next to each PVR name we provide the success rate of the corresponding policy for the given task.
• Figure 2: Standard PVR-based behaviour cloning architecture, modified with out task-progression signal. We propose our disentangling baseline, which incorporates a task-progression signal, effectively disentangling features before the policy head.
• Figure 3: Comparison of our Temporal Encoding (TE) against FLARE NEURIPS2021_ba3c5fe1 and using no temporal augmentation on PVR features. Results (sorted by TE) show (a) per-task performance and (b) per-model performance. FLARE and TE bars indicate gains over no temporal information.
• Figure 4: Correlation plots between per-PVR average policy success rate and temporal entanglement. The left plot concerns short-range temporal entanglement, as described in Section \ref{['ssec:short_range_entanglement']} and the right one concerns long-range temporal entanglement, as described in Section \ref{['ssec:long_range_entanglement']} (we omit here the non-temporal ResNets sub-group, as no trend emerged).
• Figure 5: Temporal variability in MetaWorld expert demonstrations: asynchronous task progression is evident in same-time-step frames from separate demonstrations.