VTechAGP: An Academic-to-General-Audience Text Paraphrase Dataset and Benchmark Models

TL;DR

This work introduces VTechAGP, a document-level academic-to-general-audience paraphrase benchmark derived from Virginia Tech ETDs, and proposes DSPT5, a lightweight T5-based model that uses dynamic soft prompts and a hybrid loss with crowd-sampling decoding to bridge domain knowledge gaps. Through extensive experiments across eight colleges and numerous baselines, DSPT5 demonstrates competitive performance against state-of-the-art LLMs on many metrics while offering advantages in accessibility and controllability. The study highlights trade-offs between fine-tuned lightweight models and large pre-trained LLMs across embedding-, word-, and end-to-end evaluations, and it emphasizes the need for task-specific evaluation tools for academic-to-general paraphrasing. The dataset and methods set a foundation for future improvements in domain-adaptive paraphrasing and interdisciplinary communication, with planned public release upon acceptance.

Abstract

Existing text simplification or paraphrase datasets mainly focus on sentence-level text generation in a general domain. These datasets are typically developed without using domain knowledge. In this paper, we release a novel dataset, VTechAGP, which is the first academic-to-general-audience text paraphrase dataset consisting of document-level these and dissertation academic and general-audience abstract pairs from 8 colleges authored over 25 years. We also propose a novel dynamic soft prompt generative language model, DSPT5. For training, we leverage a contrastive-generative loss function to learn the keyword vectors in the dynamic prompt. For inference, we adopt a crowd-sampling decoding strategy at both semantic and structural levels to further select the best output candidate. We evaluate DSPT5 and various state-of-the-art large language models (LLMs) from multiple perspectives. Results demonstrate that the SOTA LLMs do not provide satisfactory outcomes, while the lightweight DSPT5 can achieve competitive results. To the best of our knowledge, we are the first to build a benchmark dataset and solutions for academic-to-general-audience text paraphrase dataset. Models will be public after acceptance.

Figures (3)

• Figure 1: Dynamic soft prompt generation of our proposed model DSPT5. The framework includes a prompt encoder and a generative language model fine-tuning together with a hybrid loss. FC is a fully connected layer.
• Figure 2: The decoding strategy of crowd sampling.
• Figure 3: Human evaluation guideline.