Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Coding

TL;DR

The paper investigates how programmers interact with LLM-based conversational coding assistants through a survey, focusing on usage patterns, perceived usefulness, interaction strategies, and adoption barriers. It finds that developers primarily use these tools for debugging and explaining concepts rather than generating code, and that iterative prompt refinement and verification are common practices. Key contributions include empirical insights into usage, a set of design guidelines for context-aware, transparent, multimodal, and IDE-integrated assistants, and discussions of barriers such as trust and ethical concerns. The work advances practical guidance for integrating LLM-driven coding assistants into real software development workflows and outlines directions for future multimodal, context-preserving, and learning-supportive AI copilots.

Abstract

Conversational AI interfaces powered by large language models (LLMs) are increasingly used as coding assistants. However, questions remain about how programmers interact with LLM-based conversational agents, the challenges they encounter, and the factors influencing adoption. This study investigates programmers' usage patterns, perceptions, and interaction strategies when engaging with LLM-driven coding assistants. Through a survey, participants reported both the benefits, such as efficiency and clarity of explanations, and the limitations, including inaccuracies, lack of contextual awareness, and concerns about over-reliance. Notably, some programmers actively avoid LLMs due to a preference for independent learning, distrust in AI-generated code, and ethical considerations. Based on our findings, we propose design guidelines for improving conversational coding assistants, emphasizing context retention, transparency, multimodal support, and adaptability to user preferences. These insights contribute to the broader understanding of how LLM-based conversational agents can be effectively integrated into software development workflows while addressing adoption barriers and enhancing usability.