Categorical Keypoint Positional Embedding for Robust Animal Re-Identification

TL;DR

This work tackles wildlife re-identification under severe pose and environmental variation by combining a diffusion-based keypoint propagation pipeline with semantically enriched ViT representations. A GPT-4 guided keypoint detection step identifies discriminative landmarks on a single image, which are then propagated across the dataset via a pre-trained diffusion model, enabling robust keypoint-aware features without extensive manual labeling. The authors introduce Keypoint Positional Embedding (KPE) and Categorical Keypoint Positional Embedding (CKPE) to fuse spatial and category information of keypoints into ViT features, yielding state-of-the-art results on four wildlife benchmarks with improvements ranging from +5.9% to +50.1%. The approach reduces annotation cost, demonstrates cross-species robustness, and provides a practical, scalable pipeline for ecological monitoring; code and datasets will be released for reproducibility.

Abstract

Animal re-identification (ReID) has become an indispensable tool in ecological research, playing a critical role in tracking population dynamics, analyzing behavioral patterns, and assessing ecological impacts, all of which are vital for informed conservation strategies. Unlike human ReID, animal ReID faces significant challenges due to the high variability in animal poses, diverse environmental conditions, and the inability to directly apply pre-trained models to animal data, making the identification process across species more complex. This work introduces an innovative keypoint propagation mechanism, which utilizes a single annotated image and a pre-trained diffusion model to propagate keypoints across an entire dataset, significantly reducing the cost of manual annotation. Additionally, we enhance the Vision Transformer (ViT) by implementing Keypoint Positional Encoding (KPE) and Categorical Keypoint Positional Embedding (CKPE), enabling the ViT to learn more robust and semantically-aware representations. This provides more comprehensive and detailed keypoint representations, leading to more accurate and efficient re-identification. Our extensive experimental evaluations demonstrate that this approach significantly outperforms existing state-of-the-art methods across four wildlife datasets. The code will be publicly released.

Figures (3)

• Figure 1: Comparison of model performance between the previous state-of-the-art (SOTA) and our proposed method across four datasets. Our approach significantly outperforms the previous SOTA, demonstrating notable improvements in accuracy for all datasets.
• Figure 2: Comparison of keypoint similarity heatmaps between a pre-trained Vision Transformer (ViT) model and a pre-trained diffusion model. The most salient feature in the heatmap, indicating the keypoint, is highlighted. (a) is the image and the keypoint, above is the mouth and bottom is the right eye. (b) is the similarity heatmaps from ViT and (c) is the similarity heatmaps from Stable Diffusion. It is obvious that the pre-trained diffusion model exhibits stronger semantic correspondence compared to the pre-trained ViT model.
• Figure 3: Architecture Overview of Our Proposed Categorical Keypoint Positional Embedding for Wild Animal ReID: Initially, GPT-4 identifies critical keypoints for the ReID task from a single image. Subsequently, a diffusion model is employed to propagate these keypoints across the entire dataset. The identified keypoints then inform our ViT-based ReID module through Keypoint Positional Embedding or Categorical Keypoint Positional Embedding, focusing on the content at these specific locations and semantic information to enhance feature discrimination and accuracy.