Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs

Eilam Cohen, Itamar Bul, Danielle Inbar, Omri Loewenbach

TL;DR

This study systematically compares prompt-based and fine-tuned encoder–decoder LLMs for text simplification across three benchmarks (ASSET, Med-EASi, OneStopEnglish) using automatic metrics and human judgments. It shows a robust advantage for fine-tuning in structural simplification (SARI) while prompting often yields higher semantic similarity metrics (LENS, BERTScore) at the risk of copying, as revealed by the Identical Ratio; human evaluators also favor fine-tuned outputs. The authors release WikiLarge-Clean, prompts, and checkpoints to promote reproducibility and enable further exploration of trade-offs between data efficiency, domain adaptation, and evaluation perspectives. Overall, the work highlights a persistent trade-off between edit-rich simplification and faithfulness, suggesting a hybrid approach as a promising avenue for practical deployment. These findings have implications for selecting training paradigms in domain-sensitive text rewriting tasks and for guiding future research on scalable, high-quality simplification systems.

Abstract

Large language models (LLMs) enable strong text generation, and in general there is a practical tradeoff between fine-tuning and prompt engineering. We introduce Simplify-This, a comparative study evaluating both paradigms for text simplification with encoder-decoder LLMs across multiple benchmarks, using a range of evaluation metrics. Fine-tuned models consistently deliver stronger structural simplification, whereas prompting often attains higher semantic similarity scores yet tends to copy inputs. A human evaluation favors fine-tuned outputs overall. We release code, a cleaned derivative dataset used in our study, checkpoints of fine-tuned models, and prompt templates to facilitate reproducibility and future work.

Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs

TL;DR

This study systematically compares prompt-based and fine-tuned encoder–decoder LLMs for text simplification across three benchmarks (ASSET, Med-EASi, OneStopEnglish) using automatic metrics and human judgments. It shows a robust advantage for fine-tuning in structural simplification (SARI) while prompting often yields higher semantic similarity metrics (LENS, BERTScore) at the risk of copying, as revealed by the Identical Ratio; human evaluators also favor fine-tuned outputs. The authors release WikiLarge-Clean, prompts, and checkpoints to promote reproducibility and enable further exploration of trade-offs between data efficiency, domain adaptation, and evaluation perspectives. Overall, the work highlights a persistent trade-off between edit-rich simplification and faithfulness, suggesting a hybrid approach as a promising avenue for practical deployment. These findings have implications for selecting training paradigms in domain-sensitive text rewriting tasks and for guiding future research on scalable, high-quality simplification systems.

Abstract

Large language models (LLMs) enable strong text generation, and in general there is a practical tradeoff between fine-tuning and prompt engineering. We introduce Simplify-This, a comparative study evaluating both paradigms for text simplification with encoder-decoder LLMs across multiple benchmarks, using a range of evaluation metrics. Fine-tuned models consistently deliver stronger structural simplification, whereas prompting often attains higher semantic similarity scores yet tends to copy inputs. A human evaluation favors fine-tuned outputs overall. We release code, a cleaned derivative dataset used in our study, checkpoints of fine-tuned models, and prompt templates to facilitate reproducibility and future work.
Paper Structure (58 sections, 1 figure, 6 tables)

This paper contains 58 sections, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text simplification with LLMs
  4. Prompting vs. fine-tuning
  5. Datasets and domains
  6. Metrics and control
  7. Methodology
  8. Task
  9. Models
  10. Datasets
  11. Evaluation Datasets
  12. ASSET
  13. Med-EASi
  14. OneStopEnglish
  15. Training Dataset
  16. ...and 43 more sections

Figures (1)

  • Figure 1: Screenshot of the human evaluation interface in Qualtrics. Raters compared two candidate simplifications (order randomized) and could also select "same".