Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracing

Concentrating Solar Power (CSP) plants are a key technology in the transition toward sustainable energy. A critical factor for their safe and efficient operation is the accurate distribution of concentrated solar flux on the receiver. However, flux densities from individual heliostats are highly sensitive to surface imperfections, such as canting and mirror deformations. Measuring these surfaces across hundreds or thousands of heliostats remains impractical in real-world deployments. As a result, control systems often assume idealized heliostat surfaces, leading to suboptimal performance and potential safety risks. To address this, inverse Deep Learning Raytracing (iDLR) has recently been introduced as a novel method for inferring heliostat surfaces from target images of focal spots captured during routine calibration procedures. However, until now, iDLR had only been demonstrated in simulation. In this work, we present the first successful Sim-to-Real transfer of iDLR, enabling accurate surface predictions directly from real-world target images. Remarkably, this was achieved through a zero-shot Sim-to-Real transfer, in which the model is trained exclusively with simulated flux density data and applied directly to real target images of heliostat focal spots without the need for additional training on real target images. We evaluate our method on 63 heliostats under real operational conditions. iDLR surface predictions achieve a median mean absolute error (MAE) of only 0.17 mm and show good agreement with deflectometry ground truth in 84% of cases. When used in raytracing simulations, it enables flux density predictions with a mean accuracy of 90% compared to deflectometry over our dataset, and outperforms the commonly used ideal heliostat surface assumption by 26%. We tested this approach in a challenging double-extrapolation scenario, involving unseen sun positions and receiver projections, and found that iDLR maintains high predictive accuracy, highlighting its generalization capabilities. Our results demonstrate that iDLR is a scalable, automated, and cost-effective solution for integrating realistic heliostat surface models into digital twins. This opens the door to an optimized heliostat control, improved flux density distributions on the receiver, and ultimately, enhanced efficiency and safety in future CSP plants. • Deep learning model that predicts heliostat surfaces from target images. • Evaluation under operational conditions for 63 heliostats. • Median MAE of only 0.17 mm to Deflectometry. • Low-cost in-Situ method without new hardware and full automatization.