Adapting Feature Attenuation to NLP

TL;DR

The paper addresses the vulnerability of transformer-based NLP classifiers to inputs from unseen categories, a problem critical for deployed systems. It ports the COSTARR feature attenuation framework from computer vision to NLP by adapting it to transformer models (BERT base and GPT-2) and benchmarking against MSP, MaxLogit, and a free-energy score on a 176-category arXiv abstract task, using deployment-oriented metrics OOSA and AUOSCR. The dataset comprises 132 known and 44 unknown categories with a 10% test split, and the authors provide a detailed COSTARR formulation for transformers. Results show that COSTARR transfers to NLP without retraining but does not offer a significant gain over MaxLogit or MSP, while free-energy underperforms; this points to model capacity being a limiting factor rather than scoring strategy. The findings suggest that larger backbones and task-tailored attenuation strategies are needed to realize practical gains in Open-Set NLP, guiding future work toward more capable models and refined OSR approaches.

Abstract

Transformer classifiers such as BERT deliver impressive closed-set accuracy, yet they remain brittle when confronted with inputs from unseen categories--a common scenario for deployed NLP systems. We investigate Open-Set Recognition (OSR) for text by porting the feature attenuation hypothesis from computer vision to transformers and by benchmarking it against state-of-the-art baselines. Concretely, we adapt the COSTARR framework--originally designed for classification in computer vision--to two modest language models (BERT (base) and GPT-2) trained to label 176 arXiv subject areas. Alongside COSTARR, we evaluate Maximum Softmax Probability (MSP), MaxLogit, and the temperature-scaled free-energy score under the OOSA and AUOSCR metrics. Our results show (i) COSTARR extends to NLP without retraining but yields no statistically significant gain over MaxLogit or MSP, and (ii) free-energy lags behind all other scores in this high-class-count setting. The study highlights both the promise and the current limitations of transplanting vision-centric OSR ideas to language models, and points toward the need for larger backbones and task-tailored attenuation strategies.

Figures (5)

• Figure 1: Logit for class $j$ is formed from a dot product of its weights $W_j$ , and the input features $F$, but the low weighted features are attenuated and hence ignored by this class’s projection. The top color bar shows the weights $W$ for category 'gr-qc' from BERT (base)'s custom classification layer, sorted from low to high, left to right; the white zone denotes where weights approach zero. This sorting index orders all other color bars. The second bar shows the Hadamard Product $H$ for a known input $x_k$, and the 3rd row is for an unknown $x_u$, for which 'gr-qc' is the max logit. Each column represents a single feature dimension. The 4th and 5th bars show the feature vectors from the same test run and the most confident unknown samples from for that class, respectively. The consistency in low-saturation bars in the weights and Hadamard Product shows that regardless of what features are present, many dimensions are attenuated before classification
• Figure 2: OOSA at various thresholds for BERT (base).
• Figure 3: OOSA at various thresholds for GPT-2.
• Figure 4: OSCR curves for BERT (base).
• Figure 5: OSCR curves for GPT-2.