Software development in the age of LLMs and XR

TL;DR

This paper envisions a future in which generative AI drives coding and XR provides the dominant user interface for software development. It analyzes two primary IDE challenges—reliable code production and scalable composition—and proposes an OSS-inspired workflow where AI generates code and tests under human guidance, aided by 3D XR visualization for architectural reasoning. The authors outline a multi-stage roadmap toward zero-code development with AI-assisted loops for test and code production, plus XR-based composition and ongoing system monitoring. Key research directions include AI agent integration, remote collaboration, novel XR software metaphors, and training paths for domain experts transitioning to zero-code workflows. If realized, IDEs would become orchestration layers operating behind high-level, multimodal interfaces, potentially transforming practitioners' roles while preserving the need for development tooling.

Abstract

Let's imagine that in a few years generative AI has changed software development dramatically, taking charge of most of the programming tasks. Let's also assume that extended reality devices became ubiquitous, being the preferred interface for interacting with computers. This paper proposes how this situation would impact IDEs, by exploring how the development process would be affected, and analyzing which tools would be needed for supporting developers.