Don't be a Fool: Pooling Strategies in Offensive Language Detection from User-Intended Adversarial Attacks

TL;DR

The paper tackles robustness of offensive language detection in Korean against user-intended adversarial attacks that insert symbols or exploit Hangul features. It introduces a taxonomy of attacks (Insert, Copy, Decompose) and four layer-wise pooling strategies (mean, max, weighted, first-last) to leverage representations across all Transformer layers. Key findings show that these pooling methods improve resilience across attack rates, enabling models trained on clean text to match or exceed those pre-trained on noisy text, with first-last pooling often delivering the strongest gains. The approach is simple, does not require attackers' patterns to be explicitly trained, and exhibits potential for broader cross-lingual application and efficiency gains over ensemble methods.

Abstract

Offensive language detection is an important task for filtering out abusive expressions and improving online user experiences. However, malicious users often attempt to avoid filtering systems through the involvement of textual noises. In this paper, we propose these evasions as user-intended adversarial attacks that insert special symbols or leverage the distinctive features of the Korean language. Furthermore, we introduce simple yet effective pooling strategies in a layer-wise manner to defend against the proposed attacks, focusing on the preceding layers not just the last layer to capture both offensiveness and token embeddings. We demonstrate that these pooling strategies are more robust to performance degradation even when the attack rate is increased, without directly training of such patterns. Notably, we found that models pre-trained on clean texts could achieve a comparable performance in detecting attacked offensive language, to models pre-trained on noisy texts by employing these pooling strategies.

Figures (3)

• Figure 1: Layer-wise pooling strategies that selectively use $\left[ CLS \right]$ tokens from all layers. From the upper left, there are mean, max, weighted, and first-last pooling.
• Figure 2: Degree to which the f1-scores of the models decrease with the attack rates 30%, 60%, and 90%. We selected several baseline models and $\text{BERT}_{\textit{clean}} + first\text{-}last$ for the comparison.
• Figure 3: Initialized weights for each of the down-up and up-down poolings. Each strategy shares the shape of a cosine function but varies in the range on the x-axis depending on the layers to be focused.