Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing

Xinyu Ma, Yifeng Xu, Yang Lin, Tianlong Wang, Xu Chu, Xin Gao, Junfeng Zhao, Yasha Wang

TL;DR

DRESS addresses the challenge of stylizing LLM QA while preserving semantics by introducing train-free representation editing within a disentangled style subspace. It combines three mechanisms—attention head filtering, style subspace filtering, and adaptive editing—to locate and exploit style directions without corrupting meaning. Empirical results on Shakespearean English and Dream of the Red Chamber Chinese benchmarks show that DRESS outperforms prompting, SFT, and prior editing approaches, with notable gains in both objective and GPT-4-rated assessments. The work offers a lightweight, scalable path toward flexible stylized agents, with future directions including RAG integration and evaluation on larger models.

Abstract

We introduce DRESS, a novel approach for generating stylized large language model (LLM) responses through representation editing. Existing methods like prompting and fine-tuning are either insufficient for complex style adaptation or computationally expensive, particularly in tasks like NPC creation or character role-playing. Our approach leverages the over-parameterized nature of LLMs to disentangle a style-relevant subspace within the model's representation space to conduct representation editing, ensuring a minimal impact on the original semantics. By applying adaptive editing strengths, we dynamically adjust the steering vectors in the style subspace to maintain both stylistic fidelity and semantic integrity. We develop two stylized QA benchmark datasets to validate the effectiveness of DRESS, and the results demonstrate significant improvements compared to baseline methods such as prompting and ITI. In short, DRESS is a lightweight, train-free solution for enhancing LLMs with flexible and effective style control, making it particularly useful for developing stylized conversational agents. Codes and benchmark datasets are available at https://github.com/ArthurLeoM/DRESS-LLM.

DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing

TL;DR

DRESS addresses the challenge of stylizing LLM QA while preserving semantics by introducing train-free representation editing within a disentangled style subspace. It combines three mechanisms—attention head filtering, style subspace filtering, and adaptive editing—to locate and exploit style directions without corrupting meaning. Empirical results on Shakespearean English and Dream of the Red Chamber Chinese benchmarks show that DRESS outperforms prompting, SFT, and prior editing approaches, with notable gains in both objective and GPT-4-rated assessments. The work offers a lightweight, scalable path toward flexible stylized agents, with future directions including RAG integration and evaluation on larger models.

Abstract

We introduce DRESS, a novel approach for generating stylized large language model (LLM) responses through representation editing. Existing methods like prompting and fine-tuning are either insufficient for complex style adaptation or computationally expensive, particularly in tasks like NPC creation or character role-playing. Our approach leverages the over-parameterized nature of LLMs to disentangle a style-relevant subspace within the model's representation space to conduct representation editing, ensuring a minimal impact on the original semantics. By applying adaptive editing strengths, we dynamically adjust the steering vectors in the style subspace to maintain both stylistic fidelity and semantic integrity. We develop two stylized QA benchmark datasets to validate the effectiveness of DRESS, and the results demonstrate significant improvements compared to baseline methods such as prompting and ITI. In short, DRESS is a lightweight, train-free solution for enhancing LLMs with flexible and effective style control, making it particularly useful for developing stylized conversational agents. Codes and benchmark datasets are available at https://github.com/ArthurLeoM/DRESS-LLM.
Paper Structure (40 sections, 8 equations, 6 figures, 7 tables, 1 algorithm)

This paper contains 40 sections, 8 equations, 6 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Problem Formulation
  5. Representation Editing
  6. Methods
  7. Dataset Construction
  8. Attention Head Filtering
  9. Style Subspace Filtering
  10. Adaptive Editing
  11. Experiments
  12. Evaluation Benchmark
  13. Datasets
  14. Evaluation Metrics
  15. Baselines
  16. ...and 25 more sections

Figures (6)

  • Figure 1: An illustrative example of representation editing for Shakespeare-style responses.
  • Figure 2: The overall pipeline of DRESS. We first process the target-style QA dataset into a form suitable for solving the steering vector. Next, we use probes to filter out the attention heads most relevant to the style and further disentangle the style-related subspaces within the representation space of these heads, where the steering vectors are computed. Finally, during editing, we apply an adaptive editing strength mechanism to control the magnitude of different sub-directions in the style subspace, optimizing the editing quality while avoiding negative impacts on the output semantics.
  • Figure 3: Sensitivity analysis of varying style editing strength $\lambda$ of DRESS and ITI on Dream of the Red Chamber-style benchmark.
  • Figure 4: Sensitivity analysis on varying the number of selected attention heads $H$ of DRESS and ITI on Dream of the Red Chamber-style benchmark.
  • Figure 5: Projections of activations from target style $u^+$ and ordinary style $u^-$ to different subspaces.
  • ...and 1 more figures