DuoZone: A User-Centric, LLM-Guided Mixed-Initiative XR Window Management System

TL;DR

DuoZone addresses the cognitive and interaction burdens of XR window management by introducing two zones: an Arrangement Zone for direct manipulation of window layouts and a Recommendation Zone for LLM-guided automatic provisioning of apps and layout adjustments. The system leverages a two-stage AI workflow, combining goal-aware reasoning with local optimization to produce usable scaffolds while preserving user agency. An empirical study (N=16 experts) shows that DuoZone significantly speeds up workspace setup, reduces mental workload, and yields high acceptance of AI recommendations, while users still perform fine-tuning for precision and personal preferences. The work demonstrates a practical, scalable path toward mixed-initiative spatial computing that balances automation with user control, and it contributes open-source tooling for XR window management experiments.

Abstract

Mixed reality (XR) environments offer vast spatial possibilities, but current window management systems require users to manually place, resize, and organize multiple applications across large 3D spaces. This creates cognitive and interaction burdens that limit productivity. We introduce DuoZone, a mixed-initiative XR window management system that combines user-defined spatial layouts with LLM-guided automation. DuoZone separates window management into two complementary zones. The Recommendation Zone enables fast setup by providing spatial layout templates and automatically recommending relevant applications based on user tasks and high-level goals expressed through voice or text. The Arrangement Zone supports precise refinement through direct manipulation, allowing users to adjust windows using natural spatial actions such as dragging, resizing, and snapping. Through this dual-zone approach, DuoZone promotes efficient organization while reducing user cognitive load. We conducted a user study comparing DuoZone with a baseline manual XR window manager. Results show that DuoZone improves task completion speed, reduces mental effort, and increases sense of control when working with multiple applications in XR. We discuss design implications for future mixed-initiative systems and outline opportunities for integrating adaptive, goal-aware intelligence into spatial computing workflows.

Figures (9)

• Figure 1: DuoZone uses two spatial configurations (or zones) to achieve efficient and low cognitive load XR window management via human-AI collaboration. A Recommendation Zone (top row) offers users spatial layouts to establish areas of interest for window management swiftly. Based on established layouts, the system automatically recommends relevant applications and adjusts layouts based on users' high-level goals conveyed through voice interaction or text input. In the Arrangement Zone (bottom row), users refine the spatial layout by establishing layout space and then arranging applications using dragging, resizing, and snapping.
• Figure 2: System flow chart to indicate how different spatial configuration contributes to a mixed-initiative experience.
• Figure 3: Extending the tiled window convention, we use six different layout templates as 3D spatial anchors to support window resize, ordering, snapping, and grouping. Subfigures a) to f) indicate different tiling. For all these templates, the left upper corner tile is considered w0 and h0, with the width of the template defined as W and height as H. It is important to note that the same row cells share the same height, and the same column cells share the same width. g) An occlusion template placed in XR space prevents other zone templates from entering, allowing users to see through without virtual information blocking the view.
• Figure 4: Interactions in DuoZone’s Two-Zone Framework. In the Arrangement Zone (a–d), users directly drag, resize, rearrange, and move multiple windows to create personalized layouts. In the Recommendation Zone (e–h), users can voice requests for AI-generated layouts. The AI will automatically recommend apps, and fine-tune window sizes. Together, these modes balance user control with AI assistance for efficient workspace organization.
• Figure 5: (a–f) Participants performing interaction tasks while wearing the mixed-reality headset. Each participant manipulates virtual windows using mid-air hand gestures to complete layout configuration tasks. (g) Task 1 interface, where participants were instructed to drag and place applications into designated target locations (top, bottom, left, right) to construct layouts with varying sizes. (h) Task 2 interface where participants configured either a design or programming workspace by arranging multiple applications into an integrated, ready-to-work environment.