Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracing

TL;DR

This work demonstrates a zero-shot Sim-to-Real transfer of inverse Deep Learning Raytracing (iDLR) to real-world heliostat data, enabling surface reconstruction from focal-spot images without real training pairs. By combining Domain Adaptation (mapping real targets into the simulated domain) and Domain Randomization (extensive synthetic variability) and employing a Vision Transformer–StyleGAN2 architecture, the approach achieves a median surface error of $0.17\ \mathrm{mm}$ on 63 heliostats and maintains high flux-prediction accuracy (around $90\%$) relative to deflectometry, even under unseen sun positions. The method outperforms the traditional ideal-surface assumption by about $26\%$ in flux prediction and demonstrates strong extrapolation capability (median flux accuracy $\approx 0.90$ on a receiver scenario). These results confirm iDLR as a scalable, interpretable component for heliostat digital twins, enabling improved flux control, performance modeling, and safety in future CSP plants.

Abstract

Concentrating Solar Power (CSP) plants are a key technology in the transition toward sustainable energy. A critical factor for their safe and efficient operation is the distribution of concentrated solar flux on the receiver. However, flux distributions from individual heliostats are sensitive to surface imperfections. Measuring these surfaces across many heliostats remains impractical in real-world deployments. As a result, control systems often assume idealized heliostat surfaces, leading to suboptimal performance and potential safety risks. To address this, inverse Deep Learning Raytracing (iDLR) has been introduced as a novel method for inferring heliostat surface profiles from target images recorded during standard calibration procedures. In this work, we present the first successful Sim-to-Real transfer of iDLR, enabling accurate surface predictions directly from real-world target images. We evaluate our method on 63 heliostats under real operational conditions. iDLR surface predictions achieve a median mean absolute error (MAE) of 0.17 mm and show good agreement with deflectometry ground truth in 84% of cases. When used in raytracing simulations, it enables flux density predictions with a mean accuracy of 90% compared to deflectometry over our dataset, and outperforms the commonly used ideal heliostat surface assumption by 26%. We tested this approach in a challenging double-extrapolation scenario-involving unseen sun positions and receiver projection-and found that iDLR maintains high predictive accuracy, highlighting its generalization capabilities. Our results demonstrate that iDLR is a scalable, automated, and cost-effective solution for integrating realistic heliostat surface models into digital twins. This opens the door to improved flux control, more precise performance modeling, and ultimately, enhanced efficiency and safety in future CSP plants.

Figures (8)

• Figure 1: Overview of the full inverse Deep Learning Raytracing (iDLR) pipeline. Target images are first processed by the UNet model proposed by Kuhl2024, which translates them into flux density predictions. Up to eight such predictions are then input to the iDLR model, which has been trained solely on simulated data using Domain Randomization. The iDLR model subsequently infers the heliostat surface shape from these flux density inputs.
• Figure 2: The heliostat field of the Solar tower Jülich. The heliostat's which were used for training, validating and testing the iDLR model are highlited.
• Figure 3: The iDLR architecture consists of a Vision Transformer based Encoder and a styleGAN2 Generator.
• Figure 4: The top panel compares the iDLR surface prediction with Deflectometry measurements for ten heliostats at STJ. Above each prediction the SSIM between iDLR surface and Deflectometry is given. The lower panel compares the flux density from iDLR enhanced raytracing with Deflectometry enhanced raytracing, the UNet flux density prediction derived from a target image and the ideal heliostat assumption.
• Figure 5: The left boxplot shows the model performance without Sim-to-Real techniques, while the right boxplot shows the significantly improved performance when the full Sim-to-Real pipeline is applied.