Emotional Strain and Frustration in LLM Interactions in Software Engineering

TL;DR

The paper addresses how software engineers experience emotional strain, especially frustration, when using LLMs in professional tasks. It employs a mixed-methods, exploratory survey of 62 engineers from academia and industry and analyzes responses with qualitative content analysis aligned to Willcox's Emotions Wheel, complemented by descriptive statistics. The study identifies four main frustration triggers—incorrect/hallucinated outputs, misunderstood intent, misalignment with personal preferences, and general LLM limitations—and finds that unmet expectations can momentarily reduce motivation, though most engineers remain engaged. It offers actionable guidelines for reducing technostress through improved transparency, prompt engineering, and user training, aiming to safeguard well-being and productivity in AI-assisted software development.

Abstract

Large Language Models (LLMs) are increasingly integrated into various daily tasks in Software Engineering such as coding and requirement elicitation. Despite their various capabilities and constant use, some interactions can lead to unexpected challenges (e.g. hallucinations or verbose answers) and, in turn, cause emotions that develop into frustration. Frustration can negatively impact engineers' productivity and well-being if they escalate into stress and burnout. In this paper, we assess the impact of LLM interactions on software engineers' emotional responses, specifically strains, and identify common causes of frustration when interacting with LLMs at work. Based on 62 survey responses from software engineers in industry and academia across various companies and universities, we found that a majority of our respondents experience frustrations or other related emotions regardless of the nature of their work. Additionally, our results showed that frustration mainly stemmed from issues with correctness and less critical issues such as adaptability to context or specific format. While such issues may not cause frustration in general, artefacts that do not follow certain preferences, standards, or best practices can make the output unusable without extensive modification, causing frustration over time. In addition to the frustration triggers, our study offers guidelines to improve the software engineers' experience, aiming to minimise long-term consequences on mental health.

Figures (6)

• Figure 1: Willcox's Emotions Wheel neff2024feelings
• Figure 2: The exploratory study process that we followed to design our survey and analyse the responses qualitatively and quantitively.
• Figure 3: LLMs used by our participants.
• Figure 4: Emotional responses when receiving an incorrect answer. The colors map to Willcox’s emotions wheel in Figure \ref{['fig:wheel']}.
• Figure 5: Likert-scale results of the importance level of different aspects of the LLMs that can impact the user experience. The scale ranges from Not Important (left - red) to Extremely Important (right - green).