SynDec: A Synthesize-then-Decode Approach for Arbitrary Textual Style Transfer via Large Language Models

TL;DR

This work tackles arbitrary textual style transfer with LLMs by addressing prompt engineering and inherent stylistic biases. It introduces SynDec, a two-stage framework that first synthesize high-quality prompts via semantic-structural sampling, pattern analysis, and few-shot reranking, then amplifies their influence during decoding through contrastive decoding with a negative sample. Empirical results across six benchmarks show SynDec achieving state-of-the-art accuracy on five tasks (e.g., up to 99% for Elizabethan-to-modern transfer) and robust performance in multi-style scenarios, complemented by comprehensive ablations and expert evaluations. The approach reduces manual prompt crafting and mitigates LLM biases, offering a practical pathway for reliable arbitrary TST with large language models.

Abstract

Large Language Models (LLMs) are emerging as dominant forces for textual style transfer. However, for arbitrary style transfer, LLMs face two key challenges: (1) considerable reliance on manually-constructed prompts and (2) rigid stylistic biases inherent in LLMs. In this paper, we propose a novel Synthesize-then-Decode (SynDec) approach, which automatically synthesizes high-quality prompts and amplifies their roles during decoding process. Specifically, our approach synthesizes prompts by selecting representative few-shot samples, conducting a four-dimensional style analysis, and reranking the candidates. At LLM decoding stage, the TST effect is amplified by maximizing the contrast in output probabilities between scenarios with and without the synthesized prompt, as well as between prompts and negative samples. We conduct extensive experiments and the results show that SynDec outperforms existing state-of-the-art LLM-based methods on five out of six benchmarks (e.g., achieving up to a 9\% increase in accuracy for modern-to-Elizabethan English transfer). Detailed ablation studies further validate the effectiveness of SynDec.

Figures (5)

• Figure 1: The overview of Synthesize-then-Decode approach.
• Figure 2: Clustering results ($K$=5) on six TST datasets. Stars indicate few-shots selected by SynDec, crosses represent those obtained via random sampling, and triangles denote manually annotated representative samples.
• Figure 3: Expert evaluation of all methods on five TST tasks based on style transfer accuracy, content preservation, and fluency, with scores ranging from 0 to 5.
• Figure 4: Effect of the LLM's scope of SynDec on five TST tasks.
• Figure 5: The construction of multi-style benchmark.