A Robust Autoencoder Ensemble-Based Approach for Anomaly Detection in Text

TL;DR

This work tackles anomaly detection in text by distinguishing independent and contextual anomalies and introduces Textual Anomaly Contamination (TAC) alongside RoSAE, a Robust Subspace Local Recovery AutoEncoder Ensemble. RoSAE combines randomly connected autoencoders with a Robust Subspace Recovery layer and a Local Embedding term to learn diverse, locality-preserving latent subspaces, aggregating base detectors via a median score to improve robustness. Text representations use a RoBERTa-based Sentence Transformer rather than heavy self-attention architectures, and the approach is evaluated across eight corpora with synthetic contamination, showing strong performance on contextual anomalies and competitive robustness on independent anomalies. The work discusses limitations, notably the use of synthetic contamination, and highlights practical implications for reliably detecting nuanced text anomalies while considering ethical considerations.

Abstract

Anomaly detection (AD) is a fast growing and popular domain among established applications like vision and time series. We observe a rich literature for these applications, but anomaly detection in text is only starting to blossom. Recently, self-supervised methods with self-attention mechanism have been the most popular choice. While recent works have proposed a working ground for building and benchmarking state of the art approaches, we propose two principal contributions in this paper: contextual anomaly contamination and a novel ensemble-based approach. Our method, Textual Anomaly Contamination (TAC), allows to contaminate inlier classes with either independent or contextual anomalies. In the literature, it appears that this distinction is not performed. For finding contextual anomalies, we propose RoSAE, a Robust Subspace Local Recovery Autoencoder Ensemble. All autoencoders of the ensemble present a different latent representation through local manifold learning. Benchmark shows that our approach outperforms recent works on both independent and contextual anomalies, while being more robust. We also provide 8 dataset comparison instead of only relying to Reuters and 20 Newsgroups corpora.

Figures (6)

• Figure 1: Presentation of the studied problem with three documents topics: medical, culture and politic. Under each topic we represent a textual document with colored rectangles. Gray and green are inliers and red ones are anomalies. The detailed documents are the abstract of the news articles taken from sources like Reuters, New York times, BBC, ... The first scenario is the apparition of a culture-related document in a medical feed, and the second scenario is a political document in the culture feed.
• Figure 2: Overview of our global system, including contamination setup and our proposed approach RoSAE.
• Figure 3: Experimental study (AUC) with $\nu = 0.05$, split size of $350$ and number of base detector of $25$.
• Figure 4: Experimental study (AP) with $\nu = 0.05$, split size of $350$ and number of base detector of $25$.
• Figure 5: Results of RoSAE with contextual contamination on size of ensemble and on different values for hyperparameter K.