FANVID: A Benchmark for Face and License Plate Recognition in Low-Resolution Videos

TL;DR

FANVID addresses recognition in severely degraded LR video where single frames reveal little, requiring temporal aggregation for both faces and license plates. It introduces a video-based benchmark with LR clips downsampled to $320 \times 180$ and 20–60 FPS, annotated for identities and plate text, plus distractors to mimic real-world surveillance. The paper defines two tasks—face matching and license plate recognition—along with task-specific metrics based on $mAP@0.5$ for localization and identity/text correctness via an $EditDist$-based measure, and provides reproducible baselines using lightweight VSR (RCDM), detectors (RetinaFace, EasyOCR), and tracking (SAM2). Baselines achieve $FaceRecBox = 0.58$ and $TextRecBox = 0.42$, illustrating the viability yet challenge of LR video recognition and the need for temporally-aware architectures. FANVID offers data, tools, and evaluation scripts to spur development of efficient, privacy-conscious, temporally grounded recognition systems for surveillance, forensics, and autonomous driving.

Abstract

Real-world surveillance often renders faces and license plates unrecognizable in individual low-resolution (LR) frames, hindering reliable identification. To advance temporal recognition models, we present FANVID, a novel video-based benchmark comprising nearly 1,463 LR clips (180 x 320, 20--60 FPS) featuring 63 identities and 49 license plates from three English-speaking countries. Each video includes distractor faces and plates, increasing task difficulty and realism. The dataset contains 31,096 manually verified bounding boxes and labels. FANVID defines two tasks: (1) face matching -- detecting LR faces and matching them to high-resolution mugshots, and (2) license plate recognition -- extracting text from LR plates without a predefined database. Videos are downsampled from high-resolution sources to ensure that faces and text are indecipherable in single frames, requiring models to exploit temporal information. We introduce evaluation metrics adapted from mean Average Precision at IoU > 0.5, prioritizing identity correctness for faces and character-level accuracy for text. A baseline method with pre-trained video super-resolution, detection, and recognition achieved performance scores of 0.58 (face matching) and 0.42 (plate recognition), highlighting both the feasibility and challenge of the tasks. FANVID's selection of faces and plates balances diversity with recognition challenge. We release the software for data access, evaluation, baseline, and annotation to support reproducibility and extension. FANVID aims to catalyze innovation in temporal modeling for LR recognition, with applications in surveillance, forensics, and autonomous vehicles.

Figures (3)

• Figure 1: Bounding box dimension distribution in FANVID all LR frames (frequency is shown by bubble area) indicate that most faces and plates are too small for human recognition.
• Figure 2: Example frames with HR reference, downscaled LR (rescaled for clarity), and annotated bounding boxes for faces and license plates.
• Figure 3: Architecture of the Residual ConvNeXt Deformable Convolution with Memory (RCDM) VSR model viswanathan2024rcdm, which integrates ConvNeXt blocks, deformable alignment modules, multi-level wavelet feature extraction, and temporal memory propagation for low-resource video enhancement (reproduced with permission).