Light over Heavy: Automated Performance Requirements Quantification with Linguistic Inducement

TL;DR

This work tackles automatic quantification of performance requirements, a task traditionally done manually and prone to imprecision. It introduces LQPR, a lightweight, linguistically induced framework that reframes quantification as a classification problem and relies on structure-aware syntactic and semantic pattern matching rather than heavy data-driven learning. By extracting patterns, aligning them with requirement structures via LCS-based matching, and fusing syntactic and semantic signals with a tuned weight, LQPR achieves superior accuracy and two orders of magnitude greater efficiency than large language models across multiple real-world and synthetic datasets. The findings advocate for domain-specific, pattern-grounded approaches in software engineering tasks where concise, patterned language prevails, and they demonstrate practical benefits for configuration tuning and performance testing workflows.

Abstract

Elicited performance requirements need to be quantified for compliance in different engineering tasks, e.g., configuration tuning and performance testing. Much existing work has relied on manual quantification, which is expensive and error-prone due to the imprecision. In this paper, we present LQPR, a highly efficient automatic approach for performance requirements quantification.LQPR relies on a new theoretical framework that converts quantification as a classification problem. Despite the prevalent applications of Large Language Models (LLMs) for requirement analytics, LQPR takes a different perspective to address the classification: we observed that performance requirements can exhibit strong patterns and are often short/concise, therefore we design a lightweight linguistically induced matching mechanism. We compare LQPR against nine state-of-the-art learning-based approaches over diverse datasets, demonstrating that it is ranked as the sole best for 75% or more cases with two orders less cost. Our work proves that, at least for performance requirement quantification, specialized methods can be more suitable than the general LLM-driven approaches.

Figures (8)

• Figure 1: Performance expectations.
• Figure 2: Imprecision types.
• Figure 4: Fuzzy functions $g(v)$ that quantifies the satisfaction score of all three types of fragment. (a): $g(v)=({{s_{i}-s_{i+1}}\over{{v_i-v_{i+1}}}})v + {{s_{i+1}v_i- s_{i}v_{i+1}}\over{{v_i-v_{i+1}}}}$; (b): $g(v)=({{s_{i}-s_{i+1}}\over{{v_i-v_{i+1}}}})v + {{s_{i+1}v_i- s_{i}v_{i+1}}\over{{v_i-v_{i+1}}}}$; (c): $g(v)=s_i$.
• Figure 5: Quantification of exampled latency requirements with diverse $v_{\beta}$. Distinct fragments are colored differently.
• Figure 6: Workflow and architecture of LQPR.