Classification, Challenges, and Automated Approaches to Handle Non-Functional Requirements in ML-Enabled Systems: A Systematic Literature Review

TL;DR

The paper addresses the lack of a comprehensive synthesis of non-functional requirements (NFRs) in ML-enabled systems by performing a hybrid systematic literature review (SLR). It identifies 30 NFRs organized into six classes and catalogs over 23 software-engineering challenges, drawing from 69 primary studies gathered through a blend of conventional search, seed-set identification, and iterative snowballing. The work advances the SE4AI field by providing a detailed taxonomy, insights into ML-domain and environment-specific considerations, and implications for standards (e.g., ISO/IEC 25059) and industrial practice, including guidance for prioritizing and balancing trade-offs. The findings highlight sustainability, privacy, security, and reproducibility as critical and evolving concerns, and call for socio-technical and managerial approaches to optimize NFRs across the ML software lifecycle.

Abstract

Context: Machine learning (ML) is nowadays so pervasive and diffused that virtually no application can avoid its use. Nonetheless, its enormous potential is often tempered by the need to manage non-functional requirements and navigate pressing, contrasting trade-offs. Objective: In this respect, we notice the lack of a comprehensive synthesis of the non-functional requirements affecting ML-enabled systems, other than the major challenges faced to deal with them. Such a synthesis may not only provide a comprehensive summary of the state of the art, but also drive further research on the analysis, management, and optimization of non-functional requirements of ML-intensive systems. Method: In this paper, we propose a systematic literature review targeting two key aspects such as (1) the classification of the non-functional requirements investigated so far, and (2) the challenges to be faced when developing models in ML-enabled systems. Through the combination of well-established guidelines for conducting systematic literature reviews and additional search criteria, we survey a total amount of 69 research articles. Results: Our findings report that current research identified 30 different non-functional requirements, which can be grouped into six main classes. We also compiled a catalog of more than 23 software engineering challenges, based on which further research should consider the nonfunctional requirements of machine learning-enabled systems. Conclusion: We conclude our work by distilling implications and a future outlook on the topic.

Figures (9)

• Figure 1: Overview of the process for selecting papers.
• Figure 2: Results of the quality assessment.
• Figure 3: Papers per publication venue.
• Figure 4: Papers published per publication venue.
• Figure 5: Papers published on the matter over time.