Making the unmodulated pyramid wavefront sensor smart II. First on-sky demonstration of extreme adaptive optics with deep learning

TL;DR

This work tackles the limited linearity and sensitivity of modulated pyramid wavefront sensors (PWFS) by demonstrating, for the first time on-sky, an unmodulated PWFS controlled by a convolutional neural network (CNN) reconstructor within the MagAO-X extreme adaptive optics system. The approach leverages a trained CNN converted to a TensorRT engine to enable real-time, kilohertz-rate operation, achieving closed-loop performance that is competitive with highly optimized modulated PWFS on bright stars and even showing gains on a fainter, wind-affected target. Key findings include <2 kHz to multi-kilohertz loop operation with low latency (<250 μs single-precision, <125 μs half-precision) and Strehl ratios near those of modulated systems under favorable conditions, validating the feasibility of unmodulated PWFS with nonlinear reconstruction for XAO. The work implies substantial practical impact for next-generation instruments and future ELT-scale AO, enabling higher sensitivity and faster control without modulation hardware, while highlighting areas for improvement such as latency stability and extending performance to fainter targets.

Abstract

Pyramid wavefront sensors (PWFSs) are the preferred choice for current and future extreme adaptive optics (XAO) systems. Almost all instruments use the PWFS in its modulated form to mitigate its limited linearity range. However, this modulation comes at the cost of a reduction in sensitivity, a blindness to petal-piston modes, and a limit to the sensor's ability to operate at high speeds. Therefore, there is strong interest to use the PWFS without modulation, which can be enabled with nonlinear reconstructors. Here, we present the first on-sky demonstration of XAO with an unmodulated PWFS using a nonlinear reconstructor based on convolutional neural networks. We discuss the real-time implementation on the Magellan Adaptive Optics eXtreme (MagAO-X) instrument using the optimized TensorRT framework and show that inference is fast enough to run the control loop at >2 kHz frequencies. Our on-sky results demonstrate a successful closed-loop operation using a model calibrated with internal source data that delivers stable and robust correction under varying conditions. Performance analysis reveals that our smart PWFS achieves nearly the same Strehl ratio as the highly optimized modulated PWFS under favorable conditions on bright stars. Notably, we observe an improvement in performance on a fainter star under the influence of strong winds. These findings confirm the feasibility of using the PWFS in its unmodulated form and highlight its potential for next-generation instruments. Future efforts will focus on achieving even higher control loop frequencies (>3 kHz), optimizing the calibration procedures, and testing its performance on fainter stars, where more gain is expected for the unmodulated PWFS compared to its modulated counterpart.

Figures (4)

• Figure 1: Steps required to run the NN on-sky. First, training data is collected using the internal source by applying random shapes on the DM and recording the resulting PWFS images. After that, the NN is trained on these data and is converted to an optimized TensorRT engine. This engine is loaded in the MagAO-X control software and runs in real time, picking up the PWFS image stream and writing to the mode coefficient stream.
• Figure 2: On-sky integrated PSFs for a subset of the tests, comparing the performance between the unmodulated PWFS with the CNN reconstructor and the standard MagAO-X operation using a linear reconstructor and 3 $\lambda/D$ modulation. The estimated Strehl ratio for each of these observations is noted above the image.
• Figure 3: Measured Strehl ratio as a function of time for the test on $\alpha$ Eri on November 16, 2024, showing a generally stable performance over time. The vertical dashed red line indicates the time at which the control loop was closed. There are three frames in which the Strehl ratio drops significantly due to latency on the GPU.
• Figure 4: Latency of the convolutional part of the CNN as a function of the number of pixels across the PWFS pupil using the MagAO-X GPU. For small telescopes this is dominated by overheads, while for large telescopes we find a quadratic relationship between latency and telescope diameter.