Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data Augmentation for Fake Reviews Detection in Multiple Languages and Multiple Domains

Ming Liu, Massimo Poesio

TL;DR

This paper tackles data scarcity in fake-review detection by proposing a multilingual, multi-domain data augmentation pipeline that uses large language models (OPT, GLM-10b-Chinese) and a coherence ranker to generate synthetic reviews. The augmented data are combined with real data to train detectors (SVM and RoBERTa), with RoBERTa generally delivering superior performance; across English book, English hotels/restaurants, and Chinese DianPing datasets, augmentation yields substantial accuracy gains (up to about 11pp on Amazon and similar gains on other datasets). The authors analyze generator outputs via BLEU and coherence, finding that OPT and GLM produce higher-quality, more coherent reviews than GPT-2 and that generated data tend to resemble the seed data’s authenticity. They also discuss theoretical and practical implications, including the potential for watermarking to mitigate misuse and the need for ongoing updates to detectors as data distributions evolve. Overall, the work demonstrates that data augmentation with multilingual, domain-adapted generation can meaningfully improve fake-review detection in low-resource settings and across languages.

Abstract

With the growth of the Internet, buying habits have changed, and customers have become more dependent on the online opinions of other customers to guide their purchases. Identifying fake reviews thus became an important area for Natural Language Processing (NLP) research. However, developing high-performance NLP models depends on the availability of large amounts of training data, which are often not available for low-resource languages or domains. In this research, we used large language models to generate datasets to train fake review detectors. Our approach was used to generate fake reviews in different domains (book reviews, restaurant reviews, and hotel reviews) and different languages (English and Chinese). Our results demonstrate that our data augmentation techniques result in improved performance at fake review detection for all domains and languages. The accuracy of our fake review detection model can be improved by 0.3 percentage points on DeRev TEST, 10.9 percentage points on Amazon TEST, 8.3 percentage points on Yelp TEST and 7.2 percentage points on DianPing TEST using the augmented datasets.

Data Augmentation for Fake Reviews Detection in Multiple Languages and Multiple Domains

TL;DR

This paper tackles data scarcity in fake-review detection by proposing a multilingual, multi-domain data augmentation pipeline that uses large language models (OPT, GLM-10b-Chinese) and a coherence ranker to generate synthetic reviews. The augmented data are combined with real data to train detectors (SVM and RoBERTa), with RoBERTa generally delivering superior performance; across English book, English hotels/restaurants, and Chinese DianPing datasets, augmentation yields substantial accuracy gains (up to about 11pp on Amazon and similar gains on other datasets). The authors analyze generator outputs via BLEU and coherence, finding that OPT and GLM produce higher-quality, more coherent reviews than GPT-2 and that generated data tend to resemble the seed data’s authenticity. They also discuss theoretical and practical implications, including the potential for watermarking to mitigate misuse and the need for ongoing updates to detectors as data distributions evolve. Overall, the work demonstrates that data augmentation with multilingual, domain-adapted generation can meaningfully improve fake-review detection in low-resource settings and across languages.

Abstract

With the growth of the Internet, buying habits have changed, and customers have become more dependent on the online opinions of other customers to guide their purchases. Identifying fake reviews thus became an important area for Natural Language Processing (NLP) research. However, developing high-performance NLP models depends on the availability of large amounts of training data, which are often not available for low-resource languages or domains. In this research, we used large language models to generate datasets to train fake review detectors. Our approach was used to generate fake reviews in different domains (book reviews, restaurant reviews, and hotel reviews) and different languages (English and Chinese). Our results demonstrate that our data augmentation techniques result in improved performance at fake review detection for all domains and languages. The accuracy of our fake review detection model can be improved by 0.3 percentage points on DeRev TEST, 10.9 percentage points on Amazon TEST, 8.3 percentage points on Yelp TEST and 7.2 percentage points on DianPing TEST using the augmented datasets.

Paper Structure

This paper contains 31 sections, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Research Questions and Objectives
  3. Background
  4. Fake Reviews Detection
  5. Datasets for Fake Review Detection
  6. Data Augmentation
  7. Our Approach To Data Augmentation
  8. Methodology
  9. Generating Data: English
  10. Generalizing to Other Languages and Other Domains
  11. The Output of the Generator: An Analysis
  12. GPT-2-Generated Text
  13. OPT Generator
  14. GLM Generator
  15. Summary of the Quantitative Results
  16. ...and 16 more sections

Figures (8)

  • Figure 1: Data Augmentation. The new data obtained by augmenting the existing data is also used as training data for model training, thereby improving model efficiency.
  • Figure 2: The text generator pipeline. It contains a LLM generator used to produce texts via interpolation, and a coherence ranker to score the produced texts. It generates text of length 3, 5 or 9 after multiple iterations.
  • Figure 3: Summary of the training modalities. By combining different datasets, existing models are trained and tested to verify three questions.
  • Figure 4: The results for Amazon TEST
  • Figure 5: The results for DeRev TEST
  • ...and 3 more figures