Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dealing with Data for RE: Mitigating Challenges while using NLP and Generative AI

Smita Ghaisas, Anmol Singhal

TL;DR

The chapter tackles data-centric challenges arising from integrating NLP and Generative AI into enterprise software, with a focus on Requirements Engineering and the concept of SSNLPCore. It maps a data-centric lifecycle across data collection, annotation, processing, and validation, and offers concrete mitigation strategies and publicly available datasets to illustrate best practices. Techniques such as transfer learning, prompting, self-training, automated labeling, domain adaptation, and generative agents are highlighted, with concrete examples on datasets like PROMISE, PURE, and Unfair-TOS. The work advocates a data-centric RE framework and emphasizes interdisciplinary collaboration between data scientists and RE professionals to ensure generalizability, trustworthiness, and practical impact in real-world AI-enabled software systems.

Abstract

Across the dynamic business landscape today, enterprises face an ever-increasing range of challenges. These include the constantly evolving regulatory environment, the growing demand for personalization within software applications, and the heightened emphasis on governance. In response to these multifaceted demands, large enterprises have been adopting automation that spans from the optimization of core business processes to the enhancement of customer experiences. Indeed, Artificial Intelligence (AI) has emerged as a pivotal element of modern software systems. In this context, data plays an indispensable role. AI-centric software systems based on supervised learning and operating at an industrial scale require large volumes of training data to perform effectively. Moreover, the incorporation of generative AI has led to a growing demand for adequate evaluation benchmarks. Our experience in this field has revealed that the requirement for large datasets for training and evaluation introduces a host of intricate challenges. This book chapter explores the evolving landscape of Software Engineering (SE) in general, and Requirements Engineering (RE) in particular, in this era marked by AI integration. We discuss challenges that arise while integrating Natural Language Processing (NLP) and generative AI into enterprise-critical software systems. The chapter provides practical insights, solutions, and examples to equip readers with the knowledge and tools necessary for effectively building solutions with NLP at their cores. We also reflect on how these text data-centric tasks sit together with the traditional RE process. We also highlight new RE tasks that may be necessary for handling the increasingly important text data-centricity involved in developing software systems.

Dealing with Data for RE: Mitigating Challenges while using NLP and Generative AI

TL;DR

The chapter tackles data-centric challenges arising from integrating NLP and Generative AI into enterprise software, with a focus on Requirements Engineering and the concept of SSNLPCore. It maps a data-centric lifecycle across data collection, annotation, processing, and validation, and offers concrete mitigation strategies and publicly available datasets to illustrate best practices. Techniques such as transfer learning, prompting, self-training, automated labeling, domain adaptation, and generative agents are highlighted, with concrete examples on datasets like PROMISE, PURE, and Unfair-TOS. The work advocates a data-centric RE framework and emphasizes interdisciplinary collaboration between data scientists and RE professionals to ensure generalizability, trustworthiness, and practical impact in real-world AI-enabled software systems.

Abstract

Across the dynamic business landscape today, enterprises face an ever-increasing range of challenges. These include the constantly evolving regulatory environment, the growing demand for personalization within software applications, and the heightened emphasis on governance. In response to these multifaceted demands, large enterprises have been adopting automation that spans from the optimization of core business processes to the enhancement of customer experiences. Indeed, Artificial Intelligence (AI) has emerged as a pivotal element of modern software systems. In this context, data plays an indispensable role. AI-centric software systems based on supervised learning and operating at an industrial scale require large volumes of training data to perform effectively. Moreover, the incorporation of generative AI has led to a growing demand for adequate evaluation benchmarks. Our experience in this field has revealed that the requirement for large datasets for training and evaluation introduces a host of intricate challenges. This book chapter explores the evolving landscape of Software Engineering (SE) in general, and Requirements Engineering (RE) in particular, in this era marked by AI integration. We discuss challenges that arise while integrating Natural Language Processing (NLP) and generative AI into enterprise-critical software systems. The chapter provides practical insights, solutions, and examples to equip readers with the knowledge and tools necessary for effectively building solutions with NLP at their cores. We also reflect on how these text data-centric tasks sit together with the traditional RE process. We also highlight new RE tasks that may be necessary for handling the increasingly important text data-centricity involved in developing software systems.
Paper Structure (32 sections, 2 figures, 2 tables)

This paper contains 32 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Data-centric Stages in an SSNLPCore Lifecycle
  3. Data Collection
  4. Data Annotation
  5. Data Processing
  6. Data Validation
  7. Mitigating Data-Centric Challenges
  8. Datasets used in Exercises and/or Examples
  9. Data Collection and Annotation Stages
  10. 3.2.1 Ensuring Data Availability
  11. A) Transfer Learning:
  12. B) Prompting:
  13. C) Self-Training:
  14. D) Automated Labeling:
  15. 3.2.2 Class Imbalance
  16. ...and 17 more sections

Figures (2)

  • Figure 1: Self-Training Framework
  • Figure 2: RE in the light of Data-Centricity