Prompting in Practice: Investigating Software Developers' Use of Generative AI Tools

TL;DR

Prompting in Practice investigates how software developers real-world use GenAI tools across six SE tasks via a large-scale survey. The authors characterize usage patterns, prompting techniques, conversation strategies, and perceived reliability, revealing that code generation is nearly universal while proficiency correlates with broader task adoption and iterative multi-turn workflows are preferred. Documentation emerges as the most reliable GenAI task, whereas debugging and complex code generation pose significant reliability and output quality challenges. The study provides an empirical baseline for developer prompting practices, with actionable implications for tool design, education, and future research to improve human-AI collaboration in software engineering.

Abstract

The integration of generative artificial intelligence (GenAI) tools has fundamentally transformed software development. Although prompt engineering has emerged as a critical skill, existing research focuses primarily on individual techniques rather than software developers' broader workflows. This study presents a systematic investigation of how software engineers integrate GenAI tools into their professional practice through a large-scale survey examining prompting strategies, conversation patterns, and reliability assessments across various software engineering tasks. We surveyed 91 software engineers, including 72 active GenAI users, to understand AI usage patterns throughout the development process. Our 14 key findings show that while code generation is nearly universal, proficiency strongly correlates with using AI for more nuanced tasks such as debugging and code review, and that developers prefer iterative multi-turn conversations to single-shot prompting. Documentation tasks are perceived as most reliable, while complex code generation and debugging present sizable challenges. Our insights provide an empirical baseline of current developer practices, from simple code generation to deeper workflow integration, with actionable insights for future improvements.

Figures (7)

• Figure 1: Study Methodology (image credits at anonymous_repo)
• Figure 2: Productivity and task breadth by usage frequency
• Figure 3: Prompting technique familiarity distribution
• Figure 4: Conversation structure usage frequency
• Figure 5: Error handling strategy usage frequency