Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Robust Artificial Intelligence: Self-Supervised Learning Approach for Out-of-Distribution Detection

Wissam Salhab, Darine Ameyed, Hamid Mcheick, Fehmi Jaafar

TL;DR

This work tackles robust OOD detection without relying on labeled data by marrying self-supervised contrastive learning with graph-based clustering and Mahalanobis-distance inference. The two-phase pipeline first learns in-distribution embeddings and constructs a KNN graph whose clusters approximate local Gaussian components, then uses the nearest-cluster Mahalanobis distance for OOD scoring. Empirical results on CIFAR-10, CIFAR-100, and SVHN show state-of-the-art AUROC (up to 0.999) and strong OOD discrimination, with ablations validating the importance of graph clustering and representation learning. The approach is scalable and suitable for safety-critical applications where labeled OOD data is scarce or unavailable.

Abstract

Robustness in AI systems refers to their ability to maintain reliable and accurate performance under various conditions, including out-of-distribution (OOD) samples, adversarial attacks, and environmental changes. This is crucial in safety-critical systems, such as autonomous vehicles, transportation, or healthcare, where malfunctions could have severe consequences. This paper proposes an approach to improve OOD detection without the need of labeled data, thereby increasing the AI systems' robustness. The proposed approach leverages the principles of self-supervised learning, allowing the model to learn useful representations from unlabeled data. Combined with graph-theoretical techniques, this enables the more efficient identification and categorization of OOD samples. Compared to existing state-of-the-art methods, this approach achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) = 0.99.

Towards Robust Artificial Intelligence: Self-Supervised Learning Approach for Out-of-Distribution Detection

TL;DR

This work tackles robust OOD detection without relying on labeled data by marrying self-supervised contrastive learning with graph-based clustering and Mahalanobis-distance inference. The two-phase pipeline first learns in-distribution embeddings and constructs a KNN graph whose clusters approximate local Gaussian components, then uses the nearest-cluster Mahalanobis distance for OOD scoring. Empirical results on CIFAR-10, CIFAR-100, and SVHN show state-of-the-art AUROC (up to 0.999) and strong OOD discrimination, with ablations validating the importance of graph clustering and representation learning. The approach is scalable and suitable for safety-critical applications where labeled OOD data is scarce or unavailable.

Abstract

Robustness in AI systems refers to their ability to maintain reliable and accurate performance under various conditions, including out-of-distribution (OOD) samples, adversarial attacks, and environmental changes. This is crucial in safety-critical systems, such as autonomous vehicles, transportation, or healthcare, where malfunctions could have severe consequences. This paper proposes an approach to improve OOD detection without the need of labeled data, thereby increasing the AI systems' robustness. The proposed approach leverages the principles of self-supervised learning, allowing the model to learn useful representations from unlabeled data. Combined with graph-theoretical techniques, this enables the more efficient identification and categorization of OOD samples. Compared to existing state-of-the-art methods, this approach achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) = 0.99.
Paper Structure (31 sections, 4 equations, 3 figures, 8 tables)

This paper contains 31 sections, 4 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Mahalanobis-Based OOD Detection
  4. OOD Detection Using Supervised Classifiers
  5. OOD Detection Using Contrastive Learning
  6. OOD Detection Using Graph
  7. Limitations of Previous Methods
  8. Proposed Approach
  9. Phase 1: In-Distribution Data Representation Phase
  10. Embeddings Extraction:
  11. Graph Representation and Nodes Clustering:
  12. Phase 2: OOD Inference:
  13. Embeddings Extraction:
  14. Mahalanobis Distances Calculation:
  15. OOD Binary classification:
  16. ...and 16 more sections

Figures (3)

  • Figure 1: The proposed approach includes two phases: In-Distribution Data Representation Phase: This phase involves extracting embeddings and creating a clustered graph based on these embeddings. OOD Inference Phase: This phase consists of extracting OOD embeddings, calculating the Mahalanobis distances to the in-distribution clusters, and performing binary classification for OOD detection.
  • Figure 2: Top-5 validation accuracy chart. The model reaches a top-5 validation accuracy of 98.2% after 117000 steps for the CIFAR10 dataset. A top-5 accuracy indicates that the true labels are among the five highest predicted probabilities.
  • Figure 3: Clustered Graph: Each node in the graph represents an image embedding in latent space. We obtained 13 clusters (differentiable via colors in this image) out of 10 clusters.