DGME-T: Directional Grid Motion Encoding for Transformer-Based Historical Camera Movement Classification

TL;DR

This work tackles camera movement classification (CMC) for archival footage by first establishing a unified cross-domain benchmark that merges modern datasets into four classes and reframes HISTORIAN into five. It then introduces DGME-T, a lightweight extension to the Video Swin Transformer that injects directional grid motion priors derived from optical flow through a late fusion mechanism, with domain-aware normalization. DGME-T improves modern Top-1 accuracy from 81.78% to 86.14% and macro F1 from 82.08% to 87.81%, and also boosts HISTORIAN accuracy from 83.43% to 84.62% and macro F1 from 81.72% to 82.63%, with normalization being essential to sustain gains. A cross-domain fine-tuning stage on modern data yields further improvements of over five percentage points in historical performance, demonstrating that motion priors complement transformer representations and enhancing robustness in degraded film analysis.

Abstract

Camera movement classification (CMC) models trained on contemporary, high-quality footage often degrade when applied to archival film, where noise, missing frames, and low contrast obscure motion cues. We bridge this gap by assembling a unified benchmark that consolidates two modern corpora into four canonical classes and restructures the HISTORIAN collection into five balanced categories. Building on this benchmark, we introduce DGME-T, a lightweight extension to the Video Swin Transformer that injects directional grid motion encoding, derived from optical flow, via a learnable and normalised late-fusion layer. DGME-T raises the backbone's top-1 accuracy from 81.78% to 86.14% and its macro F1 from 82.08% to 87.81% on modern clips, while still improving the demanding World-War-II footage from 83.43% to 84.62% accuracy and from 81.72% to 82.63% macro F1. A cross-domain study further shows that an intermediate fine-tuning stage on modern data increases historical performance by more than five percentage points. These results demonstrate that structured motion priors and transformer representations are complementary and that even a small, carefully calibrated motion head can substantially enhance robustness in degraded film analysis. Related resources are available at https://github.com/linty5/DGME-T.

Figures (8)

• Figure 1: Example frames from the modern training dataset showing clean, high–resolution video content.
• Figure 2: Overall architecture of DGME-T, combining directional motion encoding with a Video Swin Transformer backbone.
• Figure 3: Global $12$-direction rose diagrams for four movement classes, shown for the modern dataset (top row, see Fig. \ref{['fig:modern_sample']}) and HISTORIAN (bottom row, see Fig. \ref{['fig:historian_teaser']}).
• Figure 4: Sample distribution of the modern dataset before and after oversampling.
• Figure 5: Class-balanced splits for the HISTORIAN dataset across five categories.