Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Curriculum-style Data Augmentation for LLM-based Metaphor Detection

Kaidi Jia, Yanxia Wu, Ming Liu, Rongsheng Li

TL;DR

This paper tackles metaphor detection under data-scarce conditions and high inference costs by fine-tuning open-source LLMs with a single inference step. It introduces Curriculum-style Data Augmentation (CDA), an iterative, teacher-guided data generation and selection process that starts with simpler data and progressively incorporates harder examples. Through a three-iteration, LoRA-finetuning pipeline using a GPT-4o teacher and Llama 3.1 8B Instruct student, the approach achieves state-of-the-art results on MOH-X and TroFi while minimizing model calls compared to GPT-3.5-based baselines. Comprehensive ablations demonstrate the effectiveness of CDA and its data-efficiency, with potential applicability to broader NLP tasks beyond metaphor detection.

Abstract

Recently, utilizing large language models (LLMs) for metaphor detection has achieved promising results. However, these methods heavily rely on the capabilities of closed-source LLMs, which come with relatively high inference costs and latency. To address this, we propose a method for metaphor detection by fine-tuning open-source LLMs, effectively reducing inference costs and latency with a single inference step. Furthermore, metaphor detection suffers from a severe data scarcity problem, which hinders effective fine-tuning of LLMs. To tackle this, we introduce Curriculum-style Data Augmentation (CDA). Specifically, before fine-tuning, we evaluate the training data to identify correctly predicted instances for fine-tuning, while incorrectly predicted instances are used as seed data for data augmentation. This approach enables the model to quickly learn simpler knowledge and progressively acquire more complex knowledge, thereby improving performance incrementally. Experimental results demonstrate that our method achieves state-of-the-art performance across all baselines. Additionally, we provide detailed ablation studies to validate the effectiveness of CDA.

Curriculum-style Data Augmentation for LLM-based Metaphor Detection

TL;DR

This paper tackles metaphor detection under data-scarce conditions and high inference costs by fine-tuning open-source LLMs with a single inference step. It introduces Curriculum-style Data Augmentation (CDA), an iterative, teacher-guided data generation and selection process that starts with simpler data and progressively incorporates harder examples. Through a three-iteration, LoRA-finetuning pipeline using a GPT-4o teacher and Llama 3.1 8B Instruct student, the approach achieves state-of-the-art results on MOH-X and TroFi while minimizing model calls compared to GPT-3.5-based baselines. Comprehensive ablations demonstrate the effectiveness of CDA and its data-efficiency, with potential applicability to broader NLP tasks beyond metaphor detection.

Abstract

Recently, utilizing large language models (LLMs) for metaphor detection has achieved promising results. However, these methods heavily rely on the capabilities of closed-source LLMs, which come with relatively high inference costs and latency. To address this, we propose a method for metaphor detection by fine-tuning open-source LLMs, effectively reducing inference costs and latency with a single inference step. Furthermore, metaphor detection suffers from a severe data scarcity problem, which hinders effective fine-tuning of LLMs. To tackle this, we introduce Curriculum-style Data Augmentation (CDA). Specifically, before fine-tuning, we evaluate the training data to identify correctly predicted instances for fine-tuning, while incorrectly predicted instances are used as seed data for data augmentation. This approach enables the model to quickly learn simpler knowledge and progressively acquire more complex knowledge, thereby improving performance incrementally. Experimental results demonstrate that our method achieves state-of-the-art performance across all baselines. Additionally, we provide detailed ablation studies to validate the effectiveness of CDA.

Paper Structure

This paper contains 27 sections, 2 figures, 18 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Metaphor Detection
  4. Data Augmentation for LLM
  5. Problem Definition
  6. Method
  7. Data Augmentation for Metaphor Detection
  8. Curriculum-style Data Augmentation
  9. Experiments
  10. Datasets
  11. Train Datasets
  12. Test Datasets
  13. Baselines
  14. Implementation Details
  15. Main Results
  16. ...and 12 more sections

Figures (2)

  • Figure 1: Structures of Our Method. The top half shows the process of Curriculum-style Data Augmentation (CDA), and the bottom half shows examples of data augmentation, metaphorical examples in blue boxes and non-metaphorical examples in green boxes.
  • Figure 2: Performance of CDA on different LLMs. This shows the performance of LLMs on the TroFi dataset across different iteration.