Convolutional Neural Networks for Toxic Comment Classification

TL;DR

This paper tackles automatic toxic comment classification in online discourse by comparing convolutional neural networks with traditional bag-of-words approaches on a Kaggle Wikipedia talk-page dataset. It uses word embeddings within a CNN architecture featuring multiple filter heights and max-over-time pooling, comparing randomly initialized versus pre-trained embeddings. The results show CNNs outperform SVM, NB, kNN, and LDA, achieving accuracies around 0.91 and high precision/recall, with CNN_fix providing the best performance and lower variance. The study suggests CNN-based embedding methods are a promising direction for scalable toxic-comment detection and points to future work on adaptive learning and n-gram extensions.

Abstract

Flood of information is produced in a daily basis through the global Internet usage arising from the on-line interactive communications among users. While this situation contributes significantly to the quality of human life, unfortunately it involves enormous dangers, since on-line texts with high toxicity can cause personal attacks, on-line harassment and bullying behaviors. This has triggered both industrial and research community in the last few years while there are several tries to identify an efficient model for on-line toxic comment prediction. However, these steps are still in their infancy and new approaches and frameworks are required. On parallel, the data explosion that appears constantly, makes the construction of new machine learning computational tools for managing this information, an imperative need. Thankfully advances in hardware, cloud computing and big data management allow the development of Deep Learning approaches appearing very promising performance so far. For text classification in particular the use of Convolutional Neural Networks (CNN) have recently been proposed approaching text analytics in a modern manner emphasizing in the structure of words in a document. In this work, we employ this approach to discover toxic comments in a large pool of documents provided by a current Kaggle's competition regarding Wikipedia's talk page edits. To justify this decision we choose to compare CNNs against the traditional bag-of-words approach for text analysis combined with a selection of algorithms proven to be very effective in text classification. The reported results provide enough evidence that CNN enhance toxic comment classification reinforcing research interest towards this direction.

Figures (4)

• Figure 1: Example of encoding a text using a vocabulary
• Figure 2: The CNN for text classification process.
• Figure 3: Two dimensional representations of the constructed DTM using PCA(right) and t-SNE(left) .
• Figure 4: Box blots for Recall, Precision and F1 score across all experiments for all Classification Methods.