A Semi-Supervised Pipeline for Generalized Behavior Discovery from Animal-Borne Motion Time Series

TL;DR

The paper tackles generalized behavior discovery in animal-borne motion time series where labels are scarce and class imbalances are severe. It introduces a semi-supervised pipeline that learns an embedding from labeled data, applies label-guided clustering to both labeled and unlabeled data with an extra cluster for novelty, and uses a KDE-based HDR containment score to quantify novelty, declaring a cluster novel when the best containment score falls below $O_c<0.3$. The approach is validated on gull IMU+GPS data across nine behaviors, using controlled discovery protocols (withheld class and negative controls) and a deployment scenario on unlabeled streams, showing that genuine novel clusters are typically identified with low containment while spurious clusters in the negative control exhibit high containment. The HDR containment statistic provides an interpretable, quantitative test for novelty in ecological motion time series under limited annotation, and the pipeline is designed to be deployment-ready for streaming data and adaptable to other short-time-series domains.

Abstract

Learning behavioral taxonomies from animal-borne sensors is challenging because labels are scarce, classes are highly imbalanced, and behaviors may be absent from the annotated set. We study generalized behavior discovery in short multivariate motion snippets from gulls, where each sample is a sequence with 3-axis IMU acceleration (20 Hz) and GPS speed, spanning nine expert-annotated behavior categories. We propose a semi-supervised discovery pipeline that (i) learns an embedding function from the labeled subset, (ii) performs label-guided clustering over embeddings of both labeled and unlabeled samples to form candidate behavior groups, and (iii) decides whether a discovered group is truly novel using a containment score. Our key contribution is a KDE + HDR (highest-density region) containment score that measures how much a discovered cluster distribution is contained within, or contains, each known-class distribution; the best-match containment score serves as an interpretable novelty statistic. In experiments where an entire behavior is withheld from supervision and appears only in the unlabeled pool, the method recovers a distinct cluster and the containment score flags novelty via low overlap, while a negative-control setting with no novel behavior yields consistently higher overlaps. These results suggest that HDR-based containment provides a practical, quantitative test for generalized class discovery in ecological motion time series under limited annotation and severe class imbalance.

Figures (10)

• Figure 1: Generalized behavior discovery from animal-borne motion. (1) Supervised representation learning: train encoder $f_\theta$ on labeled data $\mathcal{D}_L$ to embed IMU+GPS snippets (\ref{['sec:rep_learning']}). (2) Semi-supervised clustering: embed $\mathcal{D}_L \cup \mathcal{D}_U$ and run label-guided K-means to form candidate behaviors (\ref{['sec:ss-kmeans-method']}). (3) Novelty decision: compute a KDE+HDR containment score vs. known classes and flag clusters with low best-match score $O_c$ as novel (\ref{['sec:containment_score']}). Discovery: withhold one class or use only known behaviors in $\mathcal{D}_U$ (negative control)(\ref{['sec:protocols']}). Deployment: apply the same pipeline to sliding windows of unlabeled data (\ref{['sec:deployment']}).
• Figure 2: Representative existing-novel discovery run (withheld Flap). t-SNE of embeddings for 0:Flap. Top left: ground-truth labels. Top right: cluster assignments (predictions) for all data $\mathcal{D}_L \cup \mathcal{D}_U$. Bottom left: assignments restricted to labeled data $\mathcal{D}_L$. Bottom right: assignments restricted to unlabeled data $\mathcal{D}_U$. Axes show t-SNE coordinates (arbitrary units).
• Figure 3: Representative negative-control run (no novel class; removed Flap for the trial). t-SNE of embeddings with an extra cluster allocated. Top left: ground-truth labels. Top right: cluster assignments (predictions) for all data $\mathcal{D}_L \cup \mathcal{D}_U$. Bottom left: assignments restricted to labeled data $\mathcal{D}_L$. Bottom right: assignments restricted to unlabeled data $\mathcal{D}_U$. Axes show t-SNE coordinates (arbitrary units).
• Figure 4: Existing novel class discovery across all withheld behaviors. For each withheld behavior, we show a pair of t-SNE plots: ground-truth labels (left) and cluster assignments for all data$\mathcal{D}_L \cup \mathcal{D}_U$ (right). The nine withheld-behavior pairs are arranged from left to right and top to bottom in the following order: 0:Flap, 1:ExFlap, 2:Soar, 3:Boat, 4:Float, 5:SitStand, 6:TerLoco, 8:Manouvre, 9:Pecking. The border color of each pair indicates the removed class for that trial. Point colors in the left plots indicate ground-truth classes; point colors in the right plots indicate predicted cluster IDs. Axes show t-SNE coordinates (arbitrary units).
• Figure 5: Negative-control (non-existing) class discovery across trials. For each removed-class trial, we show a pair of t-SNE plots: ground-truth labels (left) and cluster assignments for all data$\mathcal{D}_L \cup \mathcal{D}_U$ (right), while still allocating one extra cluster. The nine trial pairs are arranged from left to right and top to bottom in the following order: 0:Flap, 1:ExFlap, 2:Soar, 3:Boat, 4:Float, 5:SitStand, 6:TerLoco, 8:Manouvre, 9:Pecking. The border color of each pair indicates the removed class for that trial. Point colors in the left plots indicate ground-truth classes; point colors in the right plots indicate predicted cluster IDs. Axes show t-SNE coordinates (arbitrary units).