Exploring Individual Factors in the Adoption of LLMs for Specific Software Engineering Tasks

TL;DR

This study investigates how individual technology-adoption factors shape task-specific adoption of large language models (LLMs) in software engineering. Using a two-phase survey with $N=188$ engineers and partial least squares SEM, it links UTAUT2 constructs to five LLM-use purposes: artifact manipulation ($UB1$), generating alternatives ($UB2$), information retrieval ($UB3$), decision support ($UB4$), and training ($UB5$). Results show that adoption is not uniform across tasks: habitual use and ease of use consistently drive adoption, but social influence, facilitating conditions, and other factors have task-specific, sometimes counterintuitive effects, including negative mediation for some high-ambiguity tasks. The findings offer actionable guidance for integrating LLMs into development environments and for designing task-specific AI tools that align with engineers' needs, emphasizing peer-driven knowledge sharing and seamless workflow integration. Overall, the work extends prior frameworks (e.g., Khojah et al.) by detailing antecedents of task-specific LLM adoption and providing replication data for further study.

Abstract

The advent of Large Language Models (LLMs) is transforming software development, significantly enhancing software engineering processes. Research has explored their role within development teams, focusing on specific tasks such as artifact generation, decision-making support, and information retrieval. Despite the growing body of work on LLMs in software engineering, most studies have centered on broad adoption trends, neglecting the nuanced relationship between individual cognitive and behavioral factors and their impact on task-specific adoption. While factors such as perceived effort and performance expectancy have been explored at a general level, their influence on distinct software engineering tasks remains underexamined. This gap hinders the development of tailored LLM-based systems (e.g., Generative AI Agents) that align with engineers' specific needs and limits the ability of team leaders to devise effective strategies for fostering LLM adoption in targeted workflows. This study bridges this gap by surveying N=188 software engineers to test the relationship between individual attributes related to technology adoption and LLM adoption across five key tasks, using structural equation modeling (SEM). The Unified Theory of Acceptance and Use of Technology (UTAUT2) was applied to characterize individual adoption behaviors. The findings reveal that task-specific adoption is influenced by distinct factors, some of which negatively impact adoption when considered in isolation, underscoring the complexity of LLM integration in software engineering. To support effective adoption, this article provides actionable recommendations, such as seamlessly integrating LLMs into existing development environments and encouraging peer-driven knowledge sharing to enhance information retrieval.

Figures (1)

• Figure 1: Research model and hypotheses.