Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating LLM Prompts for Data Augmentation in Multi-label Classification of Ecological Texts

Anna Glazkova, Olga Zakharova

TL;DR

This paper tackles the challenge of imbalanced data in ecological text classification by evaluating prompt-based data augmentation using an instruction-based LLM for Russian text. It compares four prompting strategies applied to the GreenRu dataset and assesses their impact on multi-label classification performance using two Russian transformers, ruELECTRA and ruBERT. The results show that LLM-driven augmentation generally improves performance over original-data baselines, with paraphrasing that clearly indicates target categories often delivering the strongest gains. The work provides practical guidance for applying LLM-generated data to address label imbalances in multilingual ecological NLP tasks and demonstrates the value of open-source Russian LLMs in this setting.

Abstract

Large language models (LLMs) play a crucial role in natural language processing (NLP) tasks, improving the understanding, generation, and manipulation of human language across domains such as translating, summarizing, and classifying text. Previous studies have demonstrated that instruction-based LLMs can be effectively utilized for data augmentation to generate diverse and realistic text samples. This study applied prompt-based data augmentation to detect mentions of green practices in Russian social media. Detecting green practices in social media aids in understanding their prevalence and helps formulate recommendations for scaling eco-friendly actions to mitigate environmental issues. We evaluated several prompts for augmenting texts in a multi-label classification task, either by rewriting existing datasets using LLMs, generating new data, or combining both approaches. Our results revealed that all strategies improved classification performance compared to the models fine-tuned only on the original dataset, outperforming baselines in most cases. The best results were obtained with the prompt that paraphrased the original text while clearly indicating the relevant categories.

Evaluating LLM Prompts for Data Augmentation in Multi-label Classification of Ecological Texts

TL;DR

This paper tackles the challenge of imbalanced data in ecological text classification by evaluating prompt-based data augmentation using an instruction-based LLM for Russian text. It compares four prompting strategies applied to the GreenRu dataset and assesses their impact on multi-label classification performance using two Russian transformers, ruELECTRA and ruBERT. The results show that LLM-driven augmentation generally improves performance over original-data baselines, with paraphrasing that clearly indicates target categories often delivering the strongest gains. The work provides practical guidance for applying LLM-generated data to address label imbalances in multilingual ecological NLP tasks and demonstrates the value of open-source Russian LLMs in this setting.

Abstract

Large language models (LLMs) play a crucial role in natural language processing (NLP) tasks, improving the understanding, generation, and manipulation of human language across domains such as translating, summarizing, and classifying text. Previous studies have demonstrated that instruction-based LLMs can be effectively utilized for data augmentation to generate diverse and realistic text samples. This study applied prompt-based data augmentation to detect mentions of green practices in Russian social media. Detecting green practices in social media aids in understanding their prevalence and helps formulate recommendations for scaling eco-friendly actions to mitigate environmental issues. We evaluated several prompts for augmenting texts in a multi-label classification task, either by rewriting existing datasets using LLMs, generating new data, or combining both approaches. Our results revealed that all strategies improved classification performance compared to the models fine-tuned only on the original dataset, outperforming baselines in most cases. The best results were obtained with the prompt that paraphrased the original text while clearly indicating the relevant categories.

Paper Structure

This paper contains 11 sections, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Data
  5. Prompting Strategies
  6. Sampling Strategies
  7. Baselines
  8. Classifiers
  9. Evaluation Metric
  10. Results and Discussion
  11. Conclusion

Figures (5)

  • Figure 1: The distribution of practices in GreenRu.
  • Figure 2: The mutual occurrence of practices in GreenRu. 1 - waste sorting, 2 - studying the product labeling, 3 - waste recycling, 4 - signing petitions, 5 - refusing purchases, 6 - exchanging, 7 - sharing, 8 - participating in actions to promote responsible consumption, 9 - repairing.
  • Figure 3: The number of mentions per practice in the training set.
  • Figure 4: Performance growth, % (ruELECTRA).
  • Figure 5: Performance growth, % (ruBERT).