Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation

Yixia Li, Boya Xiong, Guanhua Chen, Yun Chen

TL;DR

This work proposes SeTAR, a novel, training-free OOD detection method that leverages selective low-rank approximation of weight matrices in vision-language and vision-only models that enhances OOD detection via post-hoc modification of the model's weight matrices using a simple greedy search algorithm.

Abstract

Out-of-distribution (OOD) detection is crucial for the safe deployment of neural networks. Existing CLIP-based approaches perform OOD detection by devising novel scoring functions or sophisticated fine-tuning methods. In this work, we propose SeTAR, a novel, training-free OOD detection method that leverages selective low-rank approximation of weight matrices in vision-language and vision-only models. SeTAR enhances OOD detection via post-hoc modification of the model's weight matrices using a simple greedy search algorithm. Based on SeTAR, we further propose SeTAR+FT, a fine-tuning extension optimizing model performance for OOD detection tasks. Extensive evaluations on ImageNet1K and Pascal-VOC benchmarks show SeTAR's superior performance, reducing the relatively false positive rate by up to 18.95% and 36.80% compared to zero-shot and fine-tuning baselines. Ablation studies further validate SeTAR's effectiveness, robustness, and generalizability across different model backbones. Our work offers a scalable, efficient solution for OOD detection, setting a new state-of-the-art in this area.

SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation

TL;DR

This work proposes SeTAR, a novel, training-free OOD detection method that leverages selective low-rank approximation of weight matrices in vision-language and vision-only models that enhances OOD detection via post-hoc modification of the model's weight matrices using a simple greedy search algorithm.

Abstract

Out-of-distribution (OOD) detection is crucial for the safe deployment of neural networks. Existing CLIP-based approaches perform OOD detection by devising novel scoring functions or sophisticated fine-tuning methods. In this work, we propose SeTAR, a novel, training-free OOD detection method that leverages selective low-rank approximation of weight matrices in vision-language and vision-only models. SeTAR enhances OOD detection via post-hoc modification of the model's weight matrices using a simple greedy search algorithm. Based on SeTAR, we further propose SeTAR+FT, a fine-tuning extension optimizing model performance for OOD detection tasks. Extensive evaluations on ImageNet1K and Pascal-VOC benchmarks show SeTAR's superior performance, reducing the relatively false positive rate by up to 18.95% and 36.80% compared to zero-shot and fine-tuning baselines. Ablation studies further validate SeTAR's effectiveness, robustness, and generalizability across different model backbones. Our work offers a scalable, efficient solution for OOD detection, setting a new state-of-the-art in this area.
Paper Structure (49 sections, 9 equations, 7 figures, 23 tables, 1 algorithm)

This paper contains 49 sections, 9 equations, 7 figures, 23 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. CLIP Architecture
  4. Zero-shot OOD Detection with CLIP
  5. Method
  6. OOD Detection with Selective Low-Rank Approximation
  7. Low-Rank Approximation
  8. OOD Detection with Selective Low-Rank Approximation
  9. HyperParameter Search Algorithm
  10. OOD Detection with SeTAR-enhanced Low-rank Adaptation
  11. Experiments
  12. Experimental Setup
  13. Dataset
  14. Settings
  15. Metrics
  16. ...and 34 more sections

Figures (7)

  • Figure 1: The overview of SeTAR. (a) The structure of the CLIP image and text encoder. (b) The details of the feed-forward sublayer. (c) For each encoder layer, we replace the $\mathrm{W_{\text{up}}}$ weight matrix with its low-rank approximation $\mathrm{\widehat{W}_{\text{up}}}$. (d) The illustration of $\Sigma$ before and after low-rank approximation. More details are in Section \ref{['sec:lowrapx']}.
  • Figure 2: Average AUROC/FPR95 of different weight types on ImageNet1K benchmark. We use CLIP-B/16 as a backbone.
  • Figure 3: Average AUROC/FPR95 of different weight types on Pascal-VOC benchmark. We use CLIP-B/16 as a backbone.
  • Figure 4: Ablation studies on $\lambda$ on different ID datasets. We use CLIP-B/16 as a backbone.
  • Figure 5: Ablation studies on top-K on different ID datasets. We use CLIP-B/16 as a backbone.
  • ...and 2 more figures