Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review

Stefano Lambiase, Gemma Catolino, Fabio Palomba, Filomena Ferrucci

TL;DR

This paper addresses the adoption of bots and conversational agents in software engineering by conducting a multivocal literature review that combines formal and grey literature to capture both academic and practitioner perspectives. It develops a taxonomy of bot motivators, catalogs interaction, adoption, and development challenges, and identifies comprehensive best practices across development/design and interaction/adoption domains. The study also documents tangible benefits in productivity, collaboration, and technical quality, and discusses implications for human factors, training, and project management. The inclusion of an online appendix and explicit mapping between challenges and practices aims to bridge research and practice, guiding future work and practical deployment of CA-enabled SE tools.

Abstract

Bots are software systems designed to support users by automating a specific process, task, or activity. When such systems implement a conversational component to interact with the users, they are also known as conversational agents. Bots, particularly in their conversation-oriented version and AI-powered, have seen their adoption increase over time for software development and engineering purposes. Despite their exciting potential, ulteriorly enhanced by the advent of Generative AI and Large Language Models, bots still need to be improved to develop and integrate into the development cycle since practitioners report that bots add additional challenges that may worsen rather than improve. In this work, we aim to provide a taxonomy for characterizing bots, as well as a series of challenges for their adoption for Software Engineering associated with potential mitigation strategies. To reach our objectives, we conducted a multivocal literature review, reviewing both research and practitioner's literature. Through such an approach, we hope to contribute to both researchers and practitioners by providing first, a series of future research routes to follow, second, a list of strategies to adopt for improving the use of bots for software engineering purposes, and third, enforce a technology and knowledge transfer from the research field to the practitioners one, that is one of the primary goal of multivocal literature reviews.

Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review

TL;DR

This paper addresses the adoption of bots and conversational agents in software engineering by conducting a multivocal literature review that combines formal and grey literature to capture both academic and practitioner perspectives. It develops a taxonomy of bot motivators, catalogs interaction, adoption, and development challenges, and identifies comprehensive best practices across development/design and interaction/adoption domains. The study also documents tangible benefits in productivity, collaboration, and technical quality, and discusses implications for human factors, training, and project management. The inclusion of an online appendix and explicit mapping between challenges and practices aims to bridge research and practice, guiding future work and practical deployment of CA-enabled SE tools.

Abstract

Bots are software systems designed to support users by automating a specific process, task, or activity. When such systems implement a conversational component to interact with the users, they are also known as conversational agents. Bots, particularly in their conversation-oriented version and AI-powered, have seen their adoption increase over time for software development and engineering purposes. Despite their exciting potential, ulteriorly enhanced by the advent of Generative AI and Large Language Models, bots still need to be improved to develop and integrate into the development cycle since practitioners report that bots add additional challenges that may worsen rather than improve. In this work, we aim to provide a taxonomy for characterizing bots, as well as a series of challenges for their adoption for Software Engineering associated with potential mitigation strategies. To reach our objectives, we conducted a multivocal literature review, reviewing both research and practitioner's literature. Through such an approach, we hope to contribute to both researchers and practitioners by providing first, a series of future research routes to follow, second, a list of strategies to adopt for improving the use of bots for software engineering purposes, and third, enforce a technology and knowledge transfer from the research field to the practitioners one, that is one of the primary goal of multivocal literature reviews.
Paper Structure (63 sections, 2 figures, 3 tables)

This paper contains 63 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Research Study Design
  4. Planning the Review
  5. Initiation Phase
  6. Initial Examination of Previous Studies.
  7. Motivation and Needs Identification.
  8. Goals and Research Questions Definition.
  9. Search Phase
  10. Data Sources Selection.
  11. Search String Design.
  12. Eligibility Criteria Phase
  13. Conducting Phase
  14. Studies Selection Phase
  15. Data Extraction Phase
  16. ...and 48 more sections

Figures (2)

  • Figure 1: Categorization of challenges associated with bots in the context of software engineering. This categorization is based on the structure proposed by Wessel et al. wessel2021_dont_disturb_me_botChallenges. New challenges originated from our work are shown in blue.
  • Figure 2: Challenges (on the right) and Associated Best Practices (on the left).