Improved Paraphrase Generation via Controllable Latent Diffusion

TL;DR

Paraphrase generation requires producing semantically equivalent sentences in diverse forms. The paper introduces Latent Diffusion Paraphraser (LDP), which performs diffusion in the latent space of a frozen encoder–decoder bridge, linking continuous diffusion to discrete text without token-level rounding, and enables semantics-guided paraphrasing via a controllable signal. LDP uses a latent diffusion model $z_\\theta(z_t,c,t)$ conditioned on source latent $c$ with cross-attention and time-conditioning, and augments it with a trainable controller to inject keyword-based semantics ${\\hat{z}}_0 = z_\\theta(z_t,c,t) + {\\text{zero_conv}}(z'\\theta(..., c_{kw}))$, improving control over paraphrase semantics. Empirical results on QQP and Twitter show that LDP, especially with semantics guidance, achieves superior iBScore and BLEU compared to diffusion baselines and approaches or matches open large language models in quality, while delivering substantial efficiency gains by removing rounding. The approach also demonstrates domain adaptation potential and applicability to related text-generation tasks such as question generation, highlighting practical impact for scalable, controllable language generation.

Abstract

Paraphrase generation strives to generate high-quality and diverse expressions of a given text, a domain where diffusion models excel. Though SOTA diffusion generation reconciles generation quality and diversity, textual diffusion suffers from a truncation issue that hinders efficiency and quality control. In this work, we propose \textit{L}atent \textit{D}iffusion \textit{P}araphraser~(LDP), a novel paraphrase generation by modeling a controllable diffusion process given a learned latent space. LDP achieves superior generation efficiency compared to its diffusion counterparts. It can facilitate only input segments to ensure paraphrase semantics, improving the results without external features. Experiments show that LDP better reconciles paraphrase generation quality and diversity than baselines. Further analysis shows that our method is also helpful to other similar text generations and domain adaptations

Figures (3)

• Figure 1: Overview of LDP. (a) Model architecture. LDP consists of an encoder-decoder (enc-dec) framework (pink) and a diffusion module (green). The encoder (E) and decoder (D) are frozen to bridge the continuous diffusion process with corresponding discrete texts. Where the source $x$ is encoded as $c$ during the diffusion on latent variable $z$. (b) Detailed architecture of denoising model $z_\theta(\cdot)$.
• Figure 2: Architecture of LDP controller, where $c_{kw}$ indicates the encoded keyword segments. We freeze the learned $z_\theta(\cdot)$, then finetune the replica $z'_\theta(\cdot)$
• Figure 3: BLEU variance with averaged sentence length during LDP generation. The generation proceeds by diminishing noise scale with increasing BLEU and diminishing generation length.