Piloting Planetarium Visualizations with LLMs during Live Events in Science Centers

TL;DR

The paper tackles the challenge of integrating LLM-driven pilots into live planetarium shows to support human guides. It implements a two-mode OpenSpace-control system (reactive vs proactive) and conducts five real-world sessions with professional guides to evaluate performance, user experience, and interaction dynamics. Findings show that AI pilots currently struggle with nuance, pacing, and audience engagement, but can function as effective co-pilots to reduce cognitive load and enable multitasking, particularly when assets are preloaded and next actions are prepared. The work highlights practical directions for increasing autonomy, developing visualization recommender capabilities, and leveraging AI during show preparation to enhance live visualization experiences in science centers.

Abstract

We designed and evaluated an AI pilot in a planetarium visualization software, OpenSpace, for public shows in science centers. The piloting role is usually given to a human working in close collaboration with the guide on stage. We recruited 7 professional guides with extensive experience in giving shows to the public to study the impact of the AI-piloting on the overall experience. The AI-pilot is a conversational AI-agent listening to the guide and interpreting the verbal statements as commands to execute camera motions, change simulation time, or toggle visual assets. Our results show that, while AI pilots lack several critical skills for live shows, they could become useful as co-pilots to reduce workload of human pilots and allow multitasking. We propose research directions toward implementing visualization pilots and co-pilots in live settings.