How to Achieve Prototypical Birth and Death for OOD Detection?

Abstract

Out-of-Distribution (OOD) detection is crucial for the secure deployment of machine learning models, and prototype-based learning methods are among the mainstream strategies for achieving OOD detection. Existing prototype-based learning methods generally rely on a fixed number of prototypes. This static assumption fails to adapt to the inherent complexity differences across various categories. Currently, there is still a lack of a mechanism that can adaptively adjust the number of prototypes based on data complexity. Inspired by the processes of cell birth and death in biology, we propose a novel method named PID (Prototype bIrth and Death) to adaptively adjust the prototype count based on data complexity. This method relies on two dynamic mechanisms during the training process: prototype birth and prototype death. The birth mechanism instantiates new prototypes in data regions with insufficient representation by identifying the overload level of existing prototypes, thereby meticulously capturing intra-class substructures. Conversely, the death mechanism reinforces the decision boundary by pruning prototypes with ambiguous class boundaries through evaluating their discriminability. Through birth and death, the number of prototypes can be dynamically adjusted according to the data complexity, leading to the learning of more compact and better-separated In-Distribution (ID) embeddings, which significantly enhances the capability to detect OOD samples. Experiments demonstrate that our dynamic method, PID, significantly outperforms existing methods on benchmarks such as CIFAR-100, achieving State-of-the-Art (SOTA) performance, especially on the FPR95 metric.

Figures (5)

• Figure 1: Performance comparison of prototype-based versus non-prototype OOD detection methods. Purple triangles denote methods with prototype(e.g., PID, PALM), while pink squares denote methods without prototype(e.g., NPOS, VOS). The plot shows that our method achieves state-of-the-art (SOTA) performance, leading the prototype-based approach which, as a group, significantly improves AUROC and reduces FPR compared to non-prototype methods.
• Figure 2: Overview of the PID framework, illustrating one iteration of the MAP-EM algorithm. The process is divided into the E-step (top) and M-step (bottom). The E-step begins by (S1) fixing prototypes and sample representations, then (S2) computes and prunes the assignment weights. The M-step then activates: first, the Dynamic Controller (S3) evaluates the variance ($v_k^c$) and boundary score ($B_k^c$) to determine prototype birth and death. After the prototype set is updated, the assignment weights are (S4) re-calculated for the new set. Finally, (S5) performs optimization via gradient updates and non-gradient prototype updates.
• Figure 3: UMAP visualization of the first 20 subclasses of ID (CIFAR-100) and all OOD (iSUN) samples plotted to the same embedding space for methods including (a) CIDER (b) PALM (c) DMPL and (d) PID (Our). The scores are obtained by scaling the distance metrics used by each method to [0, 1] for visualization.
• Figure 4: Ablation study on the training hyperparameters of our dynamic mechanisms on CIFAR-100. (a) Sensitivity analysis of the cooldown period. (b) Sensitivity analysis of the birth and death period.
• Figure 5: The dynamic evolution of the total number of prototypes during training (Epoch 200-300).