Animal Identification with Independent Foreground and Background Modeling

TL;DR

This work tackles wildlife re-identification under domain shifts by explicitly modeling both foreground appearance and background context. It introduces Per-Instance Temperature Scaling (PITS) to calibrate per-sample predictions and fuses foreground likelihood $\Phi(x_{FG})$ with background priors $\Psi_k(x_{BG},x_{\theta})$ through $p(k|x_{FG},x_{BG},x_{\theta})$, achieving calibrated, robust identification. Background priors based on home, migrating, and time-decay locations are validated on two long-term wildlife datasets (Eurasian lynx and Caretta caretta), with substantial gains, especially for new locations. Ablation studies show calibration improvements and cross-architecture gains, though performance remains dependent on FG/BG segmentation quality and task-specific priors.

Abstract

We propose a method that robustly exploits background and foreground in visual identification of individual animals. Experiments show that their automatic separation, made easy with methods like Segment Anything, together with independent foreground and background-related modeling, improves results. The two predictions are combined in a principled way, thanks to novel Per-Instance Temperature Scaling that helps the classifier to deal with appearance ambiguities in training and to produce calibrated outputs in the inference phase. For identity prediction from the background, we propose novel spatial and temporal models. On two problems, the relative error w.r.t. the baseline was reduced by 22.3% and 8.8%, respectively. For cases where objects appear in new locations, an example of background drift, accuracy doubles.

Figures (5)

• Figure 1: Left: animal re-identification often relies on background pixels correlated with an identity frequenting a small set of locations. Center: removing the background ensures focus on the individual’s appearance but might reduce accuracy. Right: adding background in the form of prior $\Psi_k(\theta)$ and calibrating the network for each observation individually alleviates the problem and significantly improves accuracy.
• Figure 2: Left: Standard approach with Cross-Entropy loss. Right: PITS scaling to $T_i$.
• Figure 3: Examples of lynx re-identification.
• Figure 4: Examples of Loggerhead sea turtle (Caretta caretta) re-identification.
• Figure 5: Impainting location ablation. In some cases, location plays no role in identification (left), often the location is combined with individuals' appearance (middle), and in challenging cases, only the location is used to identify the individual (right).