Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An LLM-Enhanced Adversarial Editing System for Lexical Simplification

Keren Tan, Kangyang Luo, Yunshi Lan, Zheng Yuan, Jinlong Shu

TL;DR

This paper proposes a novel LS method without parallel corpora that employs an Adversarial Editing System with guidance from a confusion loss and an invariance loss to predict lexical edits in the original sentences and introduces an innovative LLM-enhanced loss to enable the distillation of knowledge from Large Language Models into a small-size LS system.

Abstract

Lexical Simplification (LS) aims to simplify text at the lexical level. Existing methods rely heavily on annotated data, making it challenging to apply in low-resource scenarios. In this paper, we propose a novel LS method without parallel corpora. This method employs an Adversarial Editing System with guidance from a confusion loss and an invariance loss to predict lexical edits in the original sentences. Meanwhile, we introduce an innovative LLM-enhanced loss to enable the distillation of knowledge from Large Language Models (LLMs) into a small-size LS system. From that, complex words within sentences are masked and a Difficulty-aware Filling module is crafted to replace masked positions with simpler words. At last, extensive experimental results and analyses on three benchmark LS datasets demonstrate the effectiveness of our proposed method.

An LLM-Enhanced Adversarial Editing System for Lexical Simplification

TL;DR

This paper proposes a novel LS method without parallel corpora that employs an Adversarial Editing System with guidance from a confusion loss and an invariance loss to predict lexical edits in the original sentences and introduces an innovative LLM-enhanced loss to enable the distillation of knowledge from Large Language Models into a small-size LS system.

Abstract

Lexical Simplification (LS) aims to simplify text at the lexical level. Existing methods rely heavily on annotated data, making it challenging to apply in low-resource scenarios. In this paper, we propose a novel LS method without parallel corpora. This method employs an Adversarial Editing System with guidance from a confusion loss and an invariance loss to predict lexical edits in the original sentences. Meanwhile, we introduce an innovative LLM-enhanced loss to enable the distillation of knowledge from Large Language Models (LLMs) into a small-size LS system. From that, complex words within sentences are masked and a Difficulty-aware Filling module is crafted to replace masked positions with simpler words. At last, extensive experimental results and analyses on three benchmark LS datasets demonstrate the effectiveness of our proposed method.
Paper Structure (15 sections, 9 equations, 2 figures, 4 tables)

This paper contains 15 sections, 9 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Task Definition and Overview
  5. Adversarial Editing
  6. Edit Predictor and Discriminator
  7. Training
  8. Difficulty-Aware Filling
  9. Experiments
  10. Experimental Settings
  11. Experiment Results
  12. Results Comparison
  13. Ablation Study
  14. Case Study
  15. Conclusion

Figures (2)

  • Figure 1: The motivation of our LLM-enhanced Adversarial Editing System, that is, distilling the knowledge from LLMs to our small-size Adversarial Editing System. In the complex sentence, correct complex words are in italic bold fonts, and red bold signifies differences between generated and original sentences.
  • Figure 2: The overall architecture of LAE-LS. Left panel: Adversarial Editing module, where an Edit Predictor predicts lexical edits guided by confusion loss, invariance loss, and LLM-enhanced loss. Right panel: Difficulty-aware Filling module, in which a Filling Module combines complex sentences and lexical edits into masked sentences as well as generates substitutions for the masked positions, aiming to simplify sentence.