A Study to Evaluate the Impact of LoRA Fine-tuning on the Performance of Non-functional Requirements Classification
Xia Li, Allen Kim
TL;DR
This study investigates the impact of LoRA-based fine-tuning in combination with prompt-based learning (p-tuning) for non-functional requirements classification (NFR) on the PROMISE dataset. Using four pre-trained models (bert-base, bert-large, roberta-base, roberta-large), the authors demonstrate substantial execution-cost reductions (up to 68%) with LoRA, while maintaining a small performance degradation of about 2–3%. The results favor RoBERTa models over BERT and show larger models benefit more from fine-tuning, indicating the practical viability of parameter-efficient fine-tuning for large-scale NFR classification. The work informs the design of efficient NLP pipelines for requirements engineering and motivates further exploration of PEFT methodologies and prompt-based templates in this domain.
Abstract
Classifying Non-Functional Requirements (NFRs) in software development life cycle is critical. Inspired by the theory of transfer learning, researchers apply powerful pre-trained models for NFR classification. However, full fine-tuning by updating all parameters of the pre-trained models is often impractical due to the huge number of parameters involved (e.g., 175 billion trainable parameters in GPT-3). In this paper, we apply Low-Rank Adaptation (LoRA) fine-tuning approach into NFR classification based on prompt-based learning to investigate its impact. The experiments show that LoRA can significantly reduce the execution cost (up to 68% reduction) without too much loss of effectiveness in classification (only 2%-3% decrease). The results show that LoRA can be practical in more complicated classification cases with larger dataset and pre-trained models.