It's Time to Get It Right: Improving Analog Clock Reading and Clock-Hand Spatial Reasoning in Vision-Language Models

TL;DR

This study introduces TickTockVQA, a human-annotated dataset containing analog clocks in diverse real-world scenarios, and proposes Swap-DPO, a direct preference optimization based fine-tuning framework to align model reasoning toward accurate time interpretation.

Abstract

Advances in vision-language models (VLMs) have achieved remarkable success on complex multimodal reasoning tasks, leading to the assumption that they should also excel at reading analog clocks. However, contrary to this expectation, our study reveals that reading analog clocks in real-world environments remains a significant challenge for state-of-the-art VLMs. Existing analog clock datasets are largely synthetic or planar with limited stylistic diversity and minimal background context, failing to capture the visual variability of real-world scenes. As a result, VLMs trained on such data exhibit weak spatial-temporal reasoning, frequently confusing the hour and minute hands and struggling under common visual conditions such as occlusion, lighting variation, and cluttered backgrounds. To address this issue, we introduce TickTockVQA, a human-annotated dataset containing analog clocks in diverse real-world scenarios. TickTockVQA provides explicit hour and minute annotations, and includes an AM/PM tag when it is inferable from the visual context. Furthermore, we propose Swap-DPO, a direct preference optimization based fine-tuning framework to align model reasoning toward accurate time interpretation. Experimental results demonstrate that our approach substantially enhances clock reading accuracy and robustness under real-world conditions, establishing a foundation for future research on spatial-temporal reasoning and visual understanding in VLMs.

Figures (11)

• Figure 1: Impact of training data quality on Qwen2.5-VL-7B performance. We compare Qwen2.5-VL-7B trained on three datasets: TickTockVQA (real-world), SynClock (OpenCV-based synthetic), and CtrlClock (diffusion-generated synthetic). Training on TickTockVQA achieves the best performance with 99.9 minutes MAE.
• Figure 2: Comparison of model predictions on the clock reading task. Our model, It's Time To Get It Right (ITGR), correctly identifies the time, while other large multimodal models (Llama-3.2-11B Zero-shot, GPT-5, Claude Sonnet 4.5, Gemini-2.5 Pro, and Perplexity Pro) produce incorrect results.
• Figure 3: Examples of challenging visual variations in the TickTockVQA test set: (a) cropped clock, (b) clock-like object, (c) illumination changes, and (d) horizontally flipped clock. The figure highlights diverse transformations and ambiguities that models must handle for robust clock understanding.
• Figure 4: Qualitative examples of hand-swap error correction by Swap-DPO. SFT incorrectly swaps the hour and minute hands, whereas Swap-DPO successfully corrects this systematic error pattern.
• Figure 5: Quantitative comparison of clock reading accuracy. Each plot visualizes the relationship between ground truth (x-axis) and model-predicted time (y-axis) in minutes. The gray dashed line ($y{=}x$) indicates perfect predictions. Left: Zero-shot baseline. Right: Our ITGR model with the Swap-DPO framework.