Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic Style Transfer for Enhancing Animal Facial Landmark Detection

Anadil Hussein, Anna Zamansky, George Martvel

TL;DR

This work addresses data scarcity in animal facial landmark detection by applying semantic neural style transfer (NST) via Splice ViT. It demonstrates that Cropped-Face Style Transfer (CF-ST) preserves facial structure more effectively than Full-Body Style Transfer (FB-ST), and introduces Supervised Style Transfer (SST) to mitigate annotation misalignment, showing that carefully selected style sources preserve accuracy. Augmenting the original CatFLW dataset with style-transferred images yields substantial improvements over baseline, with the best results obtained when combining original data with SST-generated samples (e.g., NME around 7.64 and reduced failure rates). The findings support NST as a promising, generalizable augmentation strategy for animal landmark detection and potentially other species and tasks requiring fine-grained facial analysis.

Abstract

Neural Style Transfer (NST) is a technique for applying the visual characteristics of one image onto another while preserving structural content. Traditionally used for artistic transformations, NST has recently been adapted, e.g., for domain adaptation and data augmentation. This study investigates the use of this technique for enhancing animal facial landmark detectors training. As a case study, we use a recently introduced Ensemble Landmark Detector for 48 anatomical cat facial landmarks and the CatFLW dataset it was trained on, making three main contributions. First, we demonstrate that applying style transfer to cropped facial images rather than full-body images enhances structural consistency, improving the quality of generated images. Secondly, replacing training images with style-transferred versions raised challenges of annotation misalignment, but Supervised Style Transfer (SST) - which selects style sources based on landmark accuracy - retained up to 98% of baseline accuracy. Finally, augmenting the dataset with style-transferred images further improved robustness, outperforming traditional augmentation methods. These findings establish semantic style transfer as an effective augmentation strategy for enhancing the performance of facial landmark detection models for animals and beyond. While this study focuses on cat facial landmarks, the proposed method can be generalized to other species and landmark detection models.

Semantic Style Transfer for Enhancing Animal Facial Landmark Detection

TL;DR

This work addresses data scarcity in animal facial landmark detection by applying semantic neural style transfer (NST) via Splice ViT. It demonstrates that Cropped-Face Style Transfer (CF-ST) preserves facial structure more effectively than Full-Body Style Transfer (FB-ST), and introduces Supervised Style Transfer (SST) to mitigate annotation misalignment, showing that carefully selected style sources preserve accuracy. Augmenting the original CatFLW dataset with style-transferred images yields substantial improvements over baseline, with the best results obtained when combining original data with SST-generated samples (e.g., NME around 7.64 and reduced failure rates). The findings support NST as a promising, generalizable augmentation strategy for animal landmark detection and potentially other species and tasks requiring fine-grained facial analysis.

Abstract

Neural Style Transfer (NST) is a technique for applying the visual characteristics of one image onto another while preserving structural content. Traditionally used for artistic transformations, NST has recently been adapted, e.g., for domain adaptation and data augmentation. This study investigates the use of this technique for enhancing animal facial landmark detectors training. As a case study, we use a recently introduced Ensemble Landmark Detector for 48 anatomical cat facial landmarks and the CatFLW dataset it was trained on, making three main contributions. First, we demonstrate that applying style transfer to cropped facial images rather than full-body images enhances structural consistency, improving the quality of generated images. Secondly, replacing training images with style-transferred versions raised challenges of annotation misalignment, but Supervised Style Transfer (SST) - which selects style sources based on landmark accuracy - retained up to 98% of baseline accuracy. Finally, augmenting the dataset with style-transferred images further improved robustness, outperforming traditional augmentation methods. These findings establish semantic style transfer as an effective augmentation strategy for enhancing the performance of facial landmark detection models for animals and beyond. While this study focuses on cat facial landmarks, the proposed method can be generalized to other species and landmark detection models.
Paper Structure (9 sections, 2 equations, 5 figures, 1 table)

This paper contains 9 sections, 2 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Model
  4. Dataset and Preprocessing
  5. Experimental Setup
  6. Results
  7. Training on Style-Transferred Data Alone
  8. Style-Transfer-Based Data Augmentation
  9. Conclusion

Figures (5)

  • Figure 1: The convergence behavior of the model over epochs.
  • Figure 2: Comparison between Full-Body Style Transfer (FB-ST) and Cropped-Face Style Transfer (CF-ST).
  • Figure 3: Qualitative comparison of predicted landmarks (white) and ground truth landmarks (red) across three training setups: Baseline model trained on original data (left), model trained on style-transferred data (TrainST), and model trained on a combination of original and supervised style-transferred data (Train+TrainSST(N=10)).
  • Figure 4: Comparison of Normalized Mean Error (NME) and Failure Rate (FR) across different training strategies.
  • Figure 5: Comparison of Normalized Mean Error (NME) across different facial regions among the augmentation strategies