Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data

TL;DR

This work focuses on leveraging soft-probabilities in a black-box framework, i.e. the authors can access the soft-predictions but not the internal states of the model, and shows that RAINPROOF provides OOD detection methods more aligned with task-specific performance metrics than traditional OOD detectors.

Abstract

Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from translation to chatbots, is essential. A key ingredient to ensure safe system behaviour is Out-Of-Distribution (OOD) detection, which aims to detect whether an input sample is statistically far from the training distribution. Although OOD detection is a widely covered topic in classification tasks, most methods rely on hidden features output by the encoder. In this work, we focus on leveraging soft-probabilities in a black-box framework, i.e. we can access the soft-predictions but not the internal states of the model. Our contributions include: (i) RAINPROOF a Relative informAItioN Projection OOD detection framework; and (ii) a more operational evaluation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with task-specific measures. The OOD detector may filter out samples well processed by the model and keep samples that are not, leading to weaker performance. Our results show that RAINPROOF provides OOD detection methods more aligned with task-specific performance metrics than traditional OOD detectors.

Figures (13)

• Figure 1: Ablation study on RAINPROOF for $\alpha$ and reference set size ($|\mathcal{R}|$) for dialogue shift detection. Smaller $\alpha$ emphasizes the tail of the distribution, while $\alpha=0$ counts common non-zero elements.
• Figure 2: PCA reduction of encoder's hidden features for IN and OUT distribution samples, with Mahalanobis distance mean (green cross). The plot reveals the multi-modal nature of the distributions.
• Figure 3: Effect of the temperature and $\alpha$ parameter for $a_{D_\alpha}$ on the performance on OOD detection in terms of AUROC.
• Figure 4: Impact of the temperature used to compute the energy ($a_E$) and MSP ($a_{\text{MSP}}$) OOD scores in terms of AUROC.
• Figure 5: Impact of $\alpha$ on the performance of the Rényi information projection for dialog shifts detection. A smaller $\alpha$ increases the weight of the tail of the distribution. An $\alpha$ of $0$ would consist in counting the number of the common non zero elements.

Theorems & Definitions (2)

• Remark 1
• Remark 2