StarWhisper Telescope: An AI framework for automating end-to-end astronomical observations

TL;DR

The paper tackles the bottlenecks of time-domain astronomy by introducing StarWhisper Telescope (SWT), an AI-driven framework that automates end-to-end astronomical observations across a network of telescopes. It combines large language models with modular, API-driven workflows to autonomously generate observation lists, execute real-time image analysis via the X-OPSTEP data pipeline, and trigger follow-up proposals for detected transients. The NGSS deployment demonstrates substantial gains in planning efficiency, detection latency, and multi-site coordination, while highlighting challenges in hardware automation, standardization, and system resilience. The work provides a scalable blueprint toward AI-enabled autonomy for future facilities like GOTTA and beyond, potentially transforming how large telescope networks operate and collaborate with amateur and professional astronomers alike.

Abstract

The exponential growth of large-scale telescope arrays has boosted time-domain astronomy development but introduced operational bottlenecks, including labor-intensive observation planning, data processing, and real-time decision-making. Here we present the StarWhisper Telescope system, an AI agent framework automating end-to-end astronomical observations for surveys like the Nearby Galaxy Supernovae Survey. By integrating large language models with specialized function calls and modular workflows, StarWhisper Telescope autonomously generates site-specific observation lists, executes real-time image analysis via pipelines, and dynamically triggers follow-up proposals upon transient detection. The system reduces human intervention through automated observation planning, telescope controlling and data processing, while enabling seamless collaboration between amateur and professional astronomers. Deployed across Nearby Galaxy Supernovae Survey's network of 10 amateur telescopes, the StarWhisper Telescope has detected transients with promising response times relative to existing surveys. Furthermore, StarWhisper Telescope's scalable agent architecture provides a blueprint for future facilities like the Global Open Transient Telescope Array, where AI-driven autonomy will be critical for managing 60 telescopes.

Figures (8)

• Figure 1: Workflow comparison between traditional observation methods and the SWT enhanced system. The TNS stands for Transient Name Server.
• Figure 2: The main structure of the SWT system. It begins with Observation Planning, where initial observation lists are generated. The process then moves to Agent Reporting, which incorporates updates from the Transient Name Server (TNS) report and the transients found by the data processing module. This procedure also allows manual revision and injects new targets into the observation list. These plans are then operated by the telescopes via the Observation Control module, in which the weather station can send commands to halt the observation. The Data Processing phase is designed to find transients from a data pipeline.
• Figure 3: Schematic diagram illustrating the construction of the observation list for the NGSS. Regular rectangles denote tools, while rounded rectangles represent agents powered by LLM. All orange boxes indicate input information. The constraints tool calculates both altitude and moon distance limitations. Once objects are manually added, the finalized list is saved to the server.
• Figure 4: Schematic diagram of the observation component in the NGSS. Regular rectangles denote tools, while hexagons represent hardware facilities. All orange boxes indicate input information, and green boxes correspond to output results. The observation list will be transformed into a ninaTargetSet file for N.I.N.A. to read and take control of the observation. The data will be simultaneously processed by a data pipeline. The weather station at Xinglong Observatory can automatically halt the observation by commands to N.I.N.A.. The transients found will be reported after data processing.
• Figure 5: The discovery confirmation images of AT2025pk. The images are shown from left to right are template, science, and difference image. The title contains object name (OBJNAME), background magnitude (BMAG), the pixel position of AT2025pk, Right Ascension (ra), Declination (dec), and the hour angles in hour-minute-second (hms) and degree-arcminute-arcsecond (dms).