Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ProSwitch: Knowledge-Guided Instruction Tuning to Switch Between Professional and Non-Professional Responses

Chang Zong, Yuyan Chen, Weiming Lu, Jian Shao, Yongfeng Huang, Heng Chang, Yueting Zhuang

TL;DR

ProSwitch tackles the challenge of enabling LLMs to switch between professional and non-professional responses in knowledge-intensive domains by integrating domain knowledge into a knowledge-guided instruction-tuning framework. It combines LLM-augmented data preparation, multi-level instruction formulations, and comprehensive evaluation including professionalism discrimination metrics ($Pro(O)=f_i(f_t(O,L_T), f_r(O,M))$) and reference-based quality measures, optimizing outputs with $O_p=LM(Pmt_p)$ and $O_{np}=LM(Pmt_{np})$. The approach builds domain-aware QA data from BioASQ/PubMedQA, classifies question types, balances styles, and injects term knowledge to guide tuning, yielding ProSwitch-B/T/K variants that outperform general and specialized baselines in style switching on medical and IT datasets. Human evaluation corroborates improved professionalism discrimination without sacrificing fluency or accuracy, highlighting the practical potential for tailored professional communication in domain-specific AI assistants.

Abstract

Large Language Models (LLMs) have demonstrated efficacy in various linguistic applications, including question answering and controlled text generation. However, studies into their ability to switch between opposite styles of responses in professional domains remain underexplored. This study introduces a novel approach, named ProSwitch, which enables a language model to switch between professional and non-professional answers, by tuning and evaluating through the guidance of domain and style knowledge. ProSwitch unfolds in three phases: LLM-augmented preparation to collect domain knowledge and QA pairs, instruction tuning to optimize LLMs with multiple levels of knowledge, and comprehensive evaluation to assess both style discrimination and reference-based quality of the generated text. Comparative analysis of ProSwitch against general and specialized LLMs reveals that our approach outperforms baselines in switching between professional and non-professional responses.

ProSwitch: Knowledge-Guided Instruction Tuning to Switch Between Professional and Non-Professional Responses

TL;DR

ProSwitch tackles the challenge of enabling LLMs to switch between professional and non-professional responses in knowledge-intensive domains by integrating domain knowledge into a knowledge-guided instruction-tuning framework. It combines LLM-augmented data preparation, multi-level instruction formulations, and comprehensive evaluation including professionalism discrimination metrics () and reference-based quality measures, optimizing outputs with and . The approach builds domain-aware QA data from BioASQ/PubMedQA, classifies question types, balances styles, and injects term knowledge to guide tuning, yielding ProSwitch-B/T/K variants that outperform general and specialized baselines in style switching on medical and IT datasets. Human evaluation corroborates improved professionalism discrimination without sacrificing fluency or accuracy, highlighting the practical potential for tailored professional communication in domain-specific AI assistants.

Abstract

Large Language Models (LLMs) have demonstrated efficacy in various linguistic applications, including question answering and controlled text generation. However, studies into their ability to switch between opposite styles of responses in professional domains remain underexplored. This study introduces a novel approach, named ProSwitch, which enables a language model to switch between professional and non-professional answers, by tuning and evaluating through the guidance of domain and style knowledge. ProSwitch unfolds in three phases: LLM-augmented preparation to collect domain knowledge and QA pairs, instruction tuning to optimize LLMs with multiple levels of knowledge, and comprehensive evaluation to assess both style discrimination and reference-based quality of the generated text. Comparative analysis of ProSwitch against general and specialized LLMs reveals that our approach outperforms baselines in switching between professional and non-professional responses.
Paper Structure (54 sections, 9 equations, 3 figures, 12 tables)

This paper contains 54 sections, 9 equations, 3 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text Style Transfer Learning
  4. Controllable Text Generation
  5. LLM Instruction Fine-Tuning
  6. Preliminaries
  7. Professionalism Definition
  8. Task Formulation
  9. Prompt Formulation
  10. Proposed ProSwitch
  11. LLM-Augmented Data Preparation
  12. Academic QA Pairs Collection.
  13. Question Type Classification.
  14. Data Balanced Augmentation.
  15. Term Knowledge Processing.
  16. ...and 39 more sections

Figures (3)

  • Figure 1: Two answers cater to two different types of users, where the professional one contains more technical terminology and richer semantic information.
  • Figure 2: Our ProSwitch method contains three phases to improve the style switching ability in professionalism, through exploiting domain knowledge for instruction tuning in multiple levels and performance evaluation.
  • Figure 3: Distribution of terminology count and reasoning step count from a part of PubMedPro dataset. Each value is added with a small random number for visual differentiation.