Diffusion for De-Occlusion: Accessory-Aware Diffusion Inpainting for Robust Ear Biometric Recognition

TL;DR

The paper tackles ear biometric recognition under accessory-induced occlusion by proposing a diffusion-based inpainting pre-processing pipeline tailored to ear anatomy. It automates accessory localization using a hybrid detector and SAM2 mask generation, restoring occluded regions before feeding images to Vision Transformers and evaluating across four benchmarks. Key findings show that diffusion-based restoration yields robust improvements, especially under coarse tokenization and in unconstrained data like EarVN1.0, while sometimes reducing accuracy on cleaner data, highlighting the need for identity-preserving and selective inpainting. The approach offers a practical, scalable path toward occlusion-robust ear recognition in real-world deployments, with future work on integrating restoration more tightly with recognition and extending to mixed occlusions.

Abstract

Ear occlusions (arising from the presence of ear accessories such as earrings and earphones) can negatively impact performance in ear-based biometric recognition systems, especially in unconstrained imaging circumstances. In this study, we assess the effectiveness of a diffusion-based ear inpainting technique as a pre-processing aid to mitigate the issues of ear accessory occlusions in transformer-based ear recognition systems. Given an input ear image and an automatically derived accessory mask, the inpainting model reconstructs clean and anatomically plausible ear regions by synthesizing missing pixels while preserving local geometric coherence along key ear structures, including the helix, antihelix, concha, and lobule. We evaluate the effectiveness of this pre-processing aid in transformer-based recognition systems for several vision transformer models and different patch sizes for a range of benchmark datasets. Experiments show that diffusion-based inpainting can be a useful pre-processing aid to alleviate ear accessory occlusions to improve overall recognition performance.

Figures (4)

• Figure 1: Four representative ear images (original vs. inpainted) produced by the proposed Ear-accessory Inpainting pipeline: Left indicate original aligned and cropped images (with accessories) and right indicates aligned and cropped + inpainted images (without accessories).
• Figure 2: Ear-accessory Inpainting pipeline. Ear-accessory regions are localized in the input ear image using finetuned YOLOv10 accessory detector and Grounding DINO. The resulting bounding boxes are provided to Segment Anything Model 2 (SAM2) to produce accessory segmentation masks, which are post-processed and merged into a single binary mask. The mask along with the ear image is then sent to the Inpaint Anything model which gives the final inpainted ear image.
• Figure 3: End-to-End Ear Accessory Removal and Image Restoration. From left to right: (1) input ear image, (2) detected accessory region highlighted with a red bounding box, (3) accessory segmentation mask in green, and (4) final ear image after accessory removal via inpainting. Accessory regions are detected using a fine-tuned YOLOv10 detector and Grounding DINO; the resulting boxes guide SAM2 to produce masks that are post-processed and merged into a single binary mask, which is then provided to Inpaint Anything to generate the inpainted ear image.
• Figure 4: Failure cases of inpainting. Inpainting can fail when the accessory mask is inaccurate or when black regions introduced by rotating ear images bias restoration, leading to artifacts, distorted reconstructions, and occasionally stretched ear lobules.