Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations

Yixin Liu, Kehan Yan, Shiyuan Li, Qingfeng Chen, Shirui Pan

TL;DR

This work proposes to exploit the embeddings from multiple pretrained language models and integrate them into MCA^2, a multi-view TAD framework that adopts a multi-view reconstruction model to effectively extract normal textual patterns from multiple embedding perspectives.

Abstract

Text anomaly detection (TAD) plays a critical role in various language-driven real-world applications, including harmful content moderation, phishing detection, and spam review filtering. While two-step "embedding-detector" TAD methods have shown state-of-the-art performance, their effectiveness is often limited by the use of a single embedding model and the lack of adaptability across diverse datasets and anomaly types. To address these limitations, we propose to exploit the embeddings from multiple pretrained language models and integrate them into $MCA^2$, a multi-view TAD framework. $MCA^2$ adopts a multi-view reconstruction model to effectively extract normal textual patterns from multiple embedding perspectives. To exploit inter-view complementarity, a contrastive collaboration module is designed to leverage and strengthen the interactions across different views. Moreover, an adaptive allocation module is developed to automatically assign the contribution weight of each view, thereby improving the adaptability to diverse datasets. Extensive experiments on 10 benchmark datasets verify the effectiveness of $MCA^2$ against strong baselines. The source code of $MCA^2$ is available at https://github.com/yankehan/MCA2.

Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations

TL;DR

This work proposes to exploit the embeddings from multiple pretrained language models and integrate them into MCA^2, a multi-view TAD framework that adopts a multi-view reconstruction model to effectively extract normal textual patterns from multiple embedding perspectives.

Abstract

Text anomaly detection (TAD) plays a critical role in various language-driven real-world applications, including harmful content moderation, phishing detection, and spam review filtering. While two-step "embedding-detector" TAD methods have shown state-of-the-art performance, their effectiveness is often limited by the use of a single embedding model and the lack of adaptability across diverse datasets and anomaly types. To address these limitations, we propose to exploit the embeddings from multiple pretrained language models and integrate them into , a multi-view TAD framework. adopts a multi-view reconstruction model to effectively extract normal textual patterns from multiple embedding perspectives. To exploit inter-view complementarity, a contrastive collaboration module is designed to leverage and strengthen the interactions across different views. Moreover, an adaptive allocation module is developed to automatically assign the contribution weight of each view, thereby improving the adaptability to diverse datasets. Extensive experiments on 10 benchmark datasets verify the effectiveness of against strong baselines. The source code of is available at https://github.com/yankehan/MCA2.
Paper Structure (25 sections, 11 equations, 5 figures, 5 tables, 2 algorithms)

This paper contains 25 sections, 11 equations, 5 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Methodology
  5. Multi-view Reconstruction TAD Model
  6. Inter-View Contrastive Collaboration
  7. Adaptive View Contribution Allocation
  8. Experiments
  9. Experiment Setup
  10. Main Results
  11. Ablation Study
  12. Allocation Visualization
  13. Robustness Analysis
  14. Hyperparameter Analysis
  15. Conclusion
  16. ...and 10 more sections

Figures (5)

  • Figure 1: (a) Performance comparison of different embedding models with the best detectors. (b) Visualization of embeddings on COVID-Fake dataset.
  • Figure 2: Overall framework of MCA$^2$. We illustrate the case of two views ( OpenAI and Qwen) as an example.
  • Figure 3: Distribution of the top-1 view selected by the gating module on each dataset.
  • Figure 4: Model robustness under different anomaly inject ratio (%) in inlier training data.
  • Figure 5: Hyperparameter sensitivity analysis.

Theorems & Definitions (1)

  • Definition 1: Multi-view Text Anomaly Detection