CharDiff-LP: A Diffusion Model with Character-Level Guidance for License Plate Image Restoration

TL;DR

CharDiff-LP introduces a diffusion-based license plate restoration framework that leverages character-level priors to guide inference, addressing severe degradations that impede both visual restoration and recognition. The CHARM module applies region-wise masked cross-attention to inject per-character embeddings, preventing interference across neighboring characters. Evaluations on Roboflow-LP and Dashcam-LP show improved restoration metrics (PSNR/SSIM/LPIPS) and significantly reduced CER, with a 28.3% relative CER reduction on Roboflow-LP, demonstrating the practicality of localized, character-aware guidance for robust LPR in real-world conditions.

Abstract

License plate image restoration is important not only as a preprocessing step for license plate recognition but also for enhancing evidential value, improving visual clarity, and enabling broader reuse of license plate images. We propose a novel diffusion-based framework with character-level guidance, CharDiff-LP, which effectively restores and recognizes severely degraded license plate images captured under realistic conditions. CharDiff-LP leverages fine-grained character-level priors extracted through external segmentation and Optical Character Recognition (OCR) modules tailored for low-quality license plate images. For precise and focused guidance, CharDiff-LP incorporates a novel Character-guided Attention through Region-wise Masking (CHARM) module, which ensures that each character's guidance is restricted to its own region, thereby avoiding interference with other regions. In experiments, CharDiff-LP significantly outperformed baseline restoration models in both restoration quality and recognition accuracy, achieving a 28.3% relative reduction in character error rate (CER) on the Roboflow-LP dataset compared with the best-performing baseline.

Figures (6)

• Figure 1: The results of diffusion-based image restoration models on severely degraded license plate images. Characters in red font indicate erroneously recognized and incorrectly restored characters (CER: character error rate).
• Figure 2: The overview of the CharDiff-LP framework. Character-level priors—character embeddings and spatial masks—are extracted from the low-quality input image using external segmentation and OCR modules. These priors are then injected into the diffusion model via the CHARM module to guide the restoration process at the character level throughout the denoising process.
• Figure 3: The detailed architecture of the CHARM module. Given a feature map $F_l$ from the U-Net encoder, the CHARM module injects the character embedding $e_i$ of each character only to the designated spatial region using the spatial mask $M_i$ via masked cross-attention.
• Figure 4: Restoration results on low-quality test images from the Roboflow-LP dataset. Red fonts indicate incorrectly restored characters and misrecognition results.
• Figure 5: Restoration results of low-quality test images from the Dashcam-LP dataset. Red fonts indicate incorrectly restored characters and misrecognition results.