Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Chinese Metaphor Recognition Using a Multi-stage Prompting Large Language Model

Jie Wang, Jin Wang, Xuejie Zhang

TL;DR

The paper tackles Chinese metaphor recognition, focusing on extracting tenors, vehicles, and grounds even when tenors/vehicles are not explicitly present. It introduces a two-stage generative heuristic-enhanced prompt framework: Stage I uses DeBERTa to generate candidate answers; Stage II clusters questions with k-means and creates demonstrations, which are combined into a heuristic prompt for the LLM. The method achieves competitive results on NLPCC-2024 Shared Task 9 (0.959/0.979/0.951/0.941 across tracks) and outperforms baselines in most tracks. Ablation analyses show that candidate generation and demonstration sampling contribute most to performance.

Abstract

Metaphors are common in everyday language, and the identification and understanding of metaphors are facilitated by models to achieve a better understanding of the text. Metaphors are mainly identified and generated by pre-trained models in existing research, but situations, where tenors or vehicles are not included in the metaphor, cannot be handled. The problem can be effectively solved by using Large Language Models (LLMs), but significant room for exploration remains in this early-stage research area. A multi-stage generative heuristic-enhanced prompt framework is proposed in this study to enhance the ability of LLMs to recognize tenors, vehicles, and grounds in Chinese metaphors. In the first stage, a small model is trained to obtain the required confidence score for answer candidate generation. In the second stage, questions are clustered and sampled according to specific rules. Finally, the heuristic-enhanced prompt needed is formed by combining the generated answer candidates and demonstrations. The proposed model achieved 3rd place in Track 1 of Subtask 1, 1st place in Track 2 of Subtask 1, and 1st place in both tracks of Subtask 2 at the NLPCC-2024 Shared Task 9.

Chinese Metaphor Recognition Using a Multi-stage Prompting Large Language Model

TL;DR

The paper tackles Chinese metaphor recognition, focusing on extracting tenors, vehicles, and grounds even when tenors/vehicles are not explicitly present. It introduces a two-stage generative heuristic-enhanced prompt framework: Stage I uses DeBERTa to generate candidate answers; Stage II clusters questions with k-means and creates demonstrations, which are combined into a heuristic prompt for the LLM. The method achieves competitive results on NLPCC-2024 Shared Task 9 (0.959/0.979/0.951/0.941 across tracks) and outperforms baselines in most tracks. Ablation analyses show that candidate generation and demonstration sampling contribute most to performance.

Abstract

Metaphors are common in everyday language, and the identification and understanding of metaphors are facilitated by models to achieve a better understanding of the text. Metaphors are mainly identified and generated by pre-trained models in existing research, but situations, where tenors or vehicles are not included in the metaphor, cannot be handled. The problem can be effectively solved by using Large Language Models (LLMs), but significant room for exploration remains in this early-stage research area. A multi-stage generative heuristic-enhanced prompt framework is proposed in this study to enhance the ability of LLMs to recognize tenors, vehicles, and grounds in Chinese metaphors. In the first stage, a small model is trained to obtain the required confidence score for answer candidate generation. In the second stage, questions are clustered and sampled according to specific rules. Finally, the heuristic-enhanced prompt needed is formed by combining the generated answer candidates and demonstrations. The proposed model achieved 3rd place in Track 1 of Subtask 1, 1st place in Track 2 of Subtask 1, and 1st place in both tracks of Subtask 2 at the NLPCC-2024 Shared Task 9.
Paper Structure (27 sections, 4 equations, 5 figures, 2 tables)

This paper contains 27 sections, 4 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Automated Metaphor Identification.
  4. Metaphor Recognition with LLMs.
  5. Instruction Tuning.
  6. Generative Heuristic-enhanced Prompt Framework Method
  7. Task Description
  8. Stage I: Answer Candidates Generation
  9. Stage II: Heuristics-enhanced Prompt Generation
  10. Question Clustering.
  11. Demonstration Sampling.
  12. Heuristic-enhanced Prompt Generation.
  13. Experiments
  14. Dataset and Evaluation Metrics
  15. Datasets.
  16. ...and 12 more sections

Figures (5)

  • Figure 1: The proposed approach consists of two stages: answer candidates generation and heuristic-enhanced prompt generation. In the first stage, the DeBERTa model generates a candidate list in brackets based on options and confidence scores. Then, the demonstrations are combined with the candidate list to form a heuristic-enhanced prompt.
  • Figure 2: The results of k-means clustering experiment.
  • Figure 3: Accuracy of using different prompts (a) and different sampling rules for example sampling (b).
  • Figure 4: Compare the results with the benchmark model
  • Figure 5: Examples of Communicating with Large Language Models.