Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Normative Requirements Operationalization with Large Language Models

Nick Feng, Lina Marsso, S. Getir Yaman, Isobel Standen, Yesugen Baatartogtokh, Reem Ayad, Victória Oldemburgo de Mello, Bev Townsend, Hanne Bartels, Ana Cavalcanti, Radu Calinescu, Marsha Chechik

TL;DR

This work tackles the elicitation and operationalization of normative non-functional requirements (NRs) by bridging implicit stakeholder knowledge and formal NR analysis. It introduces LLM-San, an LLM-driven pipeline that extracts Capability Relations (CR) between events and measures within the SLEEC NR formalism, and integrates these relations into the RAINCOAT end-to-end approach for elicitation, sanitization, and validation. The framework identifies new WFIs and improves the thoroughness of NR sets across real-world case studies, while also revealing practical trade-offs between systematic elicitation and automated validation. Overall, the approach has the potential to harmonize domain knowledge with formal verification techniques, enabling more robust, socially aware NR specification in complex systems such as robotics and healthcare.

Abstract

Normative non-functional requirements specify constraints that a system must observe in order to avoid violations of social, legal, ethical, empathetic, and cultural norms. As these requirements are typically defined by non-technical system stakeholders with different expertise and priorities (ethicists, lawyers, social scientists, etc.), ensuring their well-formedness and consistency is very challenging. Recent research has tackled this challenge using a domain-specific language to specify normative requirements as rules whose consistency can then be analysed with formal methods. In this paper, we propose a complementary approach that uses Large Language Models to extract semantic relationships between abstract representations of system capabilities. These relations, which are often assumed implicitly by non-technical stakeholders (e.g., based on common sense or domain knowledge), are then used to enrich the automated reasoning techniques for eliciting and analyzing the consistency of normative requirements. We show the effectiveness of our approach to normative requirements elicitation and operationalization through a range of real-world case studies.

Normative Requirements Operationalization with Large Language Models

TL;DR

This work tackles the elicitation and operationalization of normative non-functional requirements (NRs) by bridging implicit stakeholder knowledge and formal NR analysis. It introduces LLM-San, an LLM-driven pipeline that extracts Capability Relations (CR) between events and measures within the SLEEC NR formalism, and integrates these relations into the RAINCOAT end-to-end approach for elicitation, sanitization, and validation. The framework identifies new WFIs and improves the thoroughness of NR sets across real-world case studies, while also revealing practical trade-offs between systematic elicitation and automated validation. Overall, the approach has the potential to harmonize domain knowledge with formal verification techniques, enabling more robust, socially aware NR specification in complex systems such as robotics and healthcare.

Abstract

Normative non-functional requirements specify constraints that a system must observe in order to avoid violations of social, legal, ethical, empathetic, and cultural norms. As these requirements are typically defined by non-technical system stakeholders with different expertise and priorities (ethicists, lawyers, social scientists, etc.), ensuring their well-formedness and consistency is very challenging. Recent research has tackled this challenge using a domain-specific language to specify normative requirements as rules whose consistency can then be analysed with formal methods. In this paper, we propose a complementary approach that uses Large Language Models to extract semantic relationships between abstract representations of system capabilities. These relations, which are often assumed implicitly by non-technical stakeholders (e.g., based on common sense or domain knowledge), are then used to enrich the automated reasoning techniques for eliciting and analyzing the consistency of normative requirements. We show the effectiveness of our approach to normative requirements elicitation and operationalization through a range of real-world case studies.
Paper Structure (19 sections, 11 figures, 7 tables, 2 algorithms)

This paper contains 19 sections, 11 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. SLEEC
  4. Well-Formedness Issues (WFI)
  5. Sanitizing Definitions
  6. Capability Relations (CR)
  7. LLM-aided CR Extractions
  8. Relation discovery
  9. Filtering relations via logic inference
  10. The RAINCOAT Approach
  11. Evaluation
  12. Evaluation methodology
  13. RQ1
  14. RQ2
  15. Threats to validity
  16. ...and 4 more sections

Figures (11)

  • Figure 1: Overview of RAINCOAT: the noRmAtive requIremeNts eliCitatiOn and vAlidaTion approach.
  • Figure 2: Trace for the DAISY robot example.
  • Figure 3: A conflicting situation of R2 in Example \ref{['example:sc']}.
  • Figure 4: Redundancy diagnosis example.
  • Figure 5: Horn inference rules for binary relations between two events or two measures. Rules with trailing "+"s and "-"s derive positive and negative relations, respectively. Alg. \ref{['alg:consistencyChecking']} always propagates on negative ("-") before propagating on positive ("+") rules.
  • ...and 6 more figures

Theorems & Definitions (9)

  • Example 1
  • Example 2
  • Example 3
  • Example 4
  • Remark 1
  • Example 5
  • Example 6
  • Remark 2
  • Definition 1: Behaviour defined by $\textit{Rules}$