Enhancing Software Development with Context-Aware Conversational Agents: A User Study on Developer Interactions with Chatbots
Glaucia Melo, Paulo Alencar, Donald Cowan
TL;DR
This study investigates how software developers interact with context-aware conversational agents and what features they desire to augment development workflows. Using a two-phase mixed-methods design, a Rasa-based CA prototype deployed on Facebook Messenger engages 29 developers to gather quantitative and qualitative data, including questionnaires, semantic analyses, and interviews. Findings indicate strong interest in task automation, version-control support, and contextual awareness, alongside a need for explicit knowledge of capabilities and a balance between guidance and automation; several concrete design opportunities emerge, such as task portals, Git integrations, and conversation-history features. The results offer actionable guidance for designing context-aware chatbots that integrate with development tools and workflows, while outlining limitations and directions for future work on context modeling, personas, and broader deployment scenarios in software engineering.
Abstract
Software development is a cognitively intensive process requiring multitasking, adherence to evolving workflows, and continuous learning. With the rise of large language model (LLM)-based tools, such as conversational agents (CAs), there is growing interest in supporting developers through natural language interaction. However, little is known about the specific features developers seek in these systems. We conducted a user study with 29 developers using a prototype text-based chatbot to investigate preferred functionalities. Our findings reveal strong interest in task automation, version control support, and contextual adaptability, especially the need to tailor assistance for both novice and experienced users. We highlight the importance of deep contextual understanding, historical interaction awareness, and personalized support in CA design. This study contributes to the development of context-aware chatbots that enhance productivity and satisfaction, and it outlines opportunities for future research on human-AI collaboration in software engineering.