Harnessing Synthetic Preference Data for Enhancing Temporal Understanding of Video-LLMs

TL;DR

This work tackles the deficiency of Video-LLMs in fine-grained temporal understanding by introducing TimeWarp, a systematic method to generate temporally rich synthetic data and a large-scale preference dataset to fine-tune models toward temporal dynamics. TimeWarp comprises explicit data generation (TimeWarp-Explicit) using GPT-4o-Mini to create temporally focused QA pairs and implicit data generation (TimeWarp-Implicit) via an STIC-based self-training pipeline, complemented by the TimeWarMCQA benchmark to evaluate sequential event understanding. Empirical results show that models fine-tuned with TimeWarp data achieve absolute improvements across seven temporal benchmarks, with notable gains on challenging tasks (e.g., ~5% on Perception Test) and improved robustness to temporal disordering, demonstrating the value of synthetic temporal preferences for Video-LLM alignment. The findings suggest that targeted temporal data, when integrated through Direct Preference Optimization, can meaningfully elevate spatiotemporal understanding in multimodal models, paving the way for more robust video reasoning in real-world tasks. $L_{DPO}( heta, heta_{ref}) = \, \mathbb{E}_{(x, y_{w}, y_{l}) \sim S_{pref}} \left[ l\left( \lambda \log \left( \frac{p_{\theta}(y_{w}|x)}{p_{\theta_{ref}}(y_{w}|x)} \right) - \lambda \log \left( \frac{p_{\theta}(y_{l}|x)}{p_{\theta_{ref}}(y_{l}|x)} \right) \right) \right] \, (l(t) = \log(1+e^{-t}))$, and the approach demonstrates practical gains across diverse temporal reasoning tasks.

Abstract

While Video Large Language Models (Video-LLMs) have demonstrated remarkable performance across general video understanding benchmarks-particularly in video captioning and descriptive tasks-they consistently underperform on tasks that require fine-grained temporal understanding. This limitation arises due to the lack of visual complexity and temporal nuance in current fine-tuning datasets, leading these models to rely heavily on language-based reasoning rather than truly understanding video dynamics. In this work, we propose TimeWarp, a systematic method to create a targeted synthetic temporal dataset to fine-tune the model's responses to encourage it to focus on the given input video. We introduce a large-scale preference dataset, created using TimeWarp, that captures intricate temporal dynamics often overlooked, grounding the model's responses to visual and temporal information. We demonstrate that when our method is applied to existing models, it significantly improves performance on temporal understanding benchmarks, highlighting the effectiveness of our proposed datasets in advancing temporal understanding in Video-LLMs, resulting in an absolute improvement in performance across seven benchmarks. Code is available at https://github.com/sameepv21/timewarp.

Figures (9)

• Figure 1: Existing baseline methodologies such as Supervised Fine-Tuning (SFT) and base direct preference optimization (base-DPO) that do not target temporal aspects in video exhibit a deficiency in understanding temporal dynamics, leading to challenges in capturing event order. Our method (TimeWarP) improves temporal understanding in Video-LLMs. The base model used for inference here is LLaVA-Hound.
• Figure 2: An example of shuffled videos and preferred and dispreferred responses generated using our proposed method.
• Figure 3: Workflow diagram showing the use of GPT-4o-Mini for a) generating open-ended as well as multiple choice question-answer pair; b) generating dispreferred response given the same question; c) selecting option based on the shuffled captions provided for creating a benchmark for shuffled videos.
• Figure 4: 2D t-SNE visualization of question embeddings from TimeWar and seven different benchmarks, each sampling 1k random questions. Left: Shows a significant gap in current benchmarks (represented by a dotted circle). Right: Shows that TimeWar is able to fill the gap (represented by a dotted circle) providing the community with a much richer benchmark.
• Figure 5: Binary temporal order evaluation results for temporal order categories.