FPT: Feature Prompt Tuning for Few-shot Readability Assessment

TL;DR

This work tackles few-shot readability assessment by integrating rich linguistic knowledge into prompt-based tuning. The proposed Feature Prompt Tuning (FPT) embeds linguistically derived features as trainable soft prompts and introduces a calibration loss to preserve inter-class similarity, optimized via an alternating training regime. Empirical results on Chinese and English RA datasets show FPT outperforms prior prompt-based and fusion methods and even surpasses a strong LLM in several cases, highlighting the value of explicit linguistic features in prompt-tuned architectures. The approach offers a scalable, architecture-friendly pathway for leveraging handcrafted features in linguistic tasks without sole reliance on large language models.

Abstract

Prompt-based methods have achieved promising results in most few-shot text classification tasks. However, for readability assessment tasks, traditional prompt methods lackcrucial linguistic knowledge, which has already been proven to be essential. Moreover, previous studies on utilizing linguistic features have shown non-robust performance in few-shot settings and may even impair model performance.To address these issues, we propose a novel prompt-based tuning framework that incorporates rich linguistic knowledge, called Feature Prompt Tuning (FPT). Specifically, we extract linguistic features from the text and embed them into trainable soft prompts. Further, we devise a new loss function to calibrate the similarity ranking order between categories. Experimental results demonstrate that our proposed method FTP not only exhibits a significant performance improvement over the prior best prompt-based tuning approaches, but also surpasses the previous leading methods that incorporate linguistic features. Also, our proposed model significantly outperforms the large language model gpt-3.5-turbo-16k in most cases. Our proposed method establishes a new architecture for prompt tuning that sheds light on how linguistic features can be easily adapted to linguistic-related tasks.

Figures (5)

• Figure 1: Comparison of previous prompt tuning frameworks and our proposed Feature Prompt Tuning (FPT). $T(\cdot)$ and $verbalizer(\cdot)$ denote the template and verbalizer function, respectively. FPT utilizes both hard and soft tokens which are projected from the linguistic features extracted from the input $x$.
• Figure 2: The architecture of the proposed Feature Prompt Tuning. Column-wise ranking orders of similarity matrices are denoted with numbers.
• Figure 3: The comparison results of linguistic features, randomly initialized vectors and pseudo tokens.
• Figure 4: The system prompt and the user input for prompting LLM.
• Figure 5: Similarity difference matrices. We plot the difference matrices of similarity before and after linguistic feature embedding, both with and without SC. The horizontal and vertical coordinates represent the level of linguistic features. By comparing the diagonal of the matrix before and after the similarity calibration (that is, the similarity between linguistic features of the same level), the similarity between analogous categories is drawn closer.