Human-AI Experience in Integrated Development Environments: A Systematic Literature Review

TL;DR

This paper presents the first PRISMA-guided systematic literature review of in-IDE-HAX, synthesizing 90 studies identified from 2022–2024 to form a unifying perspective on Human-AI Experience inside IDEs. It introduces a three-dimensional framing—Impact, Design, and Quality—applied across two usage contexts (professional and educational) and analyzes methodologies, tasks, and lifecycle coverage. Key findings reveal productivity gains from AI-assisted coding but substantial verification overhead and safety concerns, with trust and explainability emerging as central design challenges. The work highlights Copilot as the dominant subject, identifies substantial gaps in lifecycle coverage and non-adopter perspectives, and proposes a future research agenda emphasizing longitudinal evaluations, governance, personalization, and robust verification assets to advance user-centered AI in software development tools.

Abstract

The integration of Artificial Intelligence (AI) into Integrated Development Environments (IDEs) is reshaping software development, fundamentally altering how developers interact with their tools. This shift marks the emergence of Human-AI Experience in Integrated Development Environment (in-IDE HAX), a field that explores the evolving dynamics of Human-Computer Interaction in AI-assisted coding environments. Despite rapid adoption, research on in-IDE HAX remains fragmented, which highlights the need for a unified overview of current practices, challenges, and opportunities. To provide a structured overview of existing research, we conduct a systematic literature review of 90 studies, summarizing current findings and outlining areas for further investigation. We organize key insights from reviewed studies into three aspects: Impact, Design, and Quality of AI-based systems inside IDEs. Impact findings show that AI-assisted coding enhances developer productivity but also introduces challenges, such as verification overhead and over-reliance. Design studies show that effective interfaces surface context, provide explanations and transparency of suggestion, and support user control. Quality studies document risks in correctness, maintainability, and security. For future research, priorities include productivity studies, design of assistance, and audit of AI-generated code. The agenda calls for larger and longer evaluations, stronger audit and verification assets, broader coverage across the software life cycle, and adaptive assistance under user control.

Figures (7)

• Figure 1: Flow diagram of the study.
• Figure 2: Paper IDs by Context.
• Figure 3: Paper IDs by Context and SDLC stage.
• Figure 4: Paper IDs by Types of tasks and Year.
• Figure 5: Paper Counts and IDs by Scope.