TattTRN: Template Reconstruction Network for Tattoo Retrieval

TL;DR

TattTRN tackles tattoo retrieval under data scarcity by introducing a large semi-synthetic tattoo database (28,550 images across 571 templates) and a template-based retrieval network. The Tattoo Template Reconstruction Network learns to map input tattoo samples to clean templates via synthetic data generation, image-to-template translation, and dual embedding backbones whose features are concatenated for final similarity scoring, optimized with ArcFace losses and a reconstruction term. Empirical results on WebTattoo and BIVTatt show strong closed-set and competitive open-set performance, with rank-1 IR up to 81.60% (WebTattoo) and 96.54% (BIVTatt), and up to 95% IR by rank-20 when using the ITT component. The approach outperforms baselines trained on real data, demonstrating the viability of semi-synthetic data for forensic tattoo retrieval and offering practical utility for identifying suspects within limited candidate lists; future work includes semantic prompt integration and skin-tone effects in tattoo segmentation and retrieval.

Abstract

Tattoos have been used effectively as soft biometrics to assist law enforcement in the identification of offenders and victims, as they contain discriminative information, and are a useful indicator to locate members of a criminal gang or organisation. Due to various privacy issues in the acquisition of images containing tattoos, only a limited number of databases exists. This lack of databases has delayed the development of new methods to effectively retrieve a potential suspect's tattoo images from a candidate gallery. To mitigate this issue, in our work, we use an unsupervised generative approach to create a balanced database consisting of 28,550 semi-synthetic images with tattooed subjects from 571 tattoo categories. Further, we introduce a novel Tattoo Template Reconstruction Network (TattTRN), which learns to map the input tattoo sample to its respective tattoo template to enhance the distinguishing attributes of the final feature embedding. Experimental results with real data, i.e., WebTattoo and BIVTatt databases, demonstrate the soundness of the presented approach: an accuracy of up to 99% is achieved for checking at most the first 20 entries of the candidate list.

Figures (8)

• Figure 1: Examples of tattoos from WebTattoo Han-TattooImgSearch-PAMI-2019 representing a challenge for state-of-the-art solutions.
• Figure 2: Conceptual overview of the proposed system: the template transformed from the input image helps to mitigate challenges related to the capturing process. The final feature embedding representing the salient properties of the input tattoo is the concatenation of the two computed feature embeddings and can be used to retrieve similar tattoo samples in a database.
• Figure 3: Examples of tattoos generated on chest and back images with their respective segmentation maps (\ref{['fig:back_generated']}) using base images and random tattoo templates (\ref{['fig:back_orig']}). Cropped tattoo images used in the network training (\ref{['fig:cropped_back']}).
• Figure 4: Examples of the database used to evaluate the proposed TattTRN system.
• Figure 5: CMC curves for different backbones combined with TattTRN.