Segmentation and Celestial Mapping of Unobservable Regions in Nighttime All-sky Images for the Mephisto Observations

TL;DR

The paper addresses the challenge of real-time visibility assessment in nighttime astronomy by developing a deep learning-based segmentation framework that yields pixel-level masks of clouds and moonlit regions from all-sky images. The Enhanced UNet, featuring an EfficientNet-B4 encoder and SCSE attention with a hybrid BCE/Dice/IoU loss, achieves an IoU of $0.9212$, Precision of $0.9564$, Recall of $0.9602$, and F1 of $0.9537$ on a 2,000-image nighttime dataset. A Zenithal Equal-Area projection-based two-stage coordinate mapping enables precise conversion from telescope pointing (RA/Dec) to image pixels, with a mean residual of $0.95$ pixels, allowing real-time cloud-aware scheduling in the Mephisto OCS. The framework is designed to generalize to other wide-field robotic observatories and forms a foundation for autonomous, weather-aware sky surveys, with future work on temporal cloud modeling and broader deployment.

Abstract

Accurate identification of unobservable regions in nighttime is essential for autonomous scheduling and data quality control in observations.Traditional methods-such as infrared sensing or photometric extinction-provide only coarse,non-spatial estimates of sky clarity,making them insufficient for real-time decision-making.This not only wastes observing time but also introduces contamination when telescopes are directed toward cloud-covered or moonlight-affected regions.To address these limitations,we propose a deep learning-based segmentation framework that provides pixel-level masks of unobservable areas using all-sky images.Supported by a manually annotated dataset of nighttime images,our method enables precise detection of cloud- and moonlight-affected regions.The segmentation results are further mapped to celestial coordinates through Zenithal Equal-Area projection,allowing seamless integration with observation control systems (OCS) for real-time cloud-aware scheduling.While developed for the Mephisto telescope,the framework is generalizable and applicable to other wide-field robotic observatories equipped with all-sky monitoring.

Figures (8)

• Figure 1: Image preprocessing workflow: (a) Raw image captured by the all-sky camera; (b) Cropped and resized circular region; (c) Annotated image with masked obstructions; (d) Binarized mask highlighting cloud objects (white) and background (black).
• Figure 2: Examples of full sky image and mask image in three categories. Two representative samples are shown for each category. The mask ratio is given at the bottom for each sample.
• Figure 3: The architecture of the proposed Enhanced UNet. The encoder is based on EfficientNet-B4, while the decoder incorporates SCSE attention modules for feature enhancement. Skip connections bridge the encoder and decoder to retain critical spatial details and contextual information. The output layer applies a Sigmoid activation for pixel-level precision.
• Figure 4: Performance evaluation of the U-Net baseline and Enhanced UNet configurations on the expanded dataset. The Enhanced UNet achieves consistently better IoU, Precision, Recall, and F1 Score. The configuration ($w_1 = 0.4, w_2 = 0.2, w_3 = 0.4$) yields the best results.
• Figure 5: Visualization of model performance for three categories (High, Moderate, and Low Unobservable Region). From top to bottom, the panels show: the raw sky image; the model prediction mask; the ground truth mask; the overlay image (showing the overlap of model prediction and ground truth on the raw image); and the difference map. The difference map highlights discrepancies between the model prediction and ground truth: red indicates areas incorrectly predicted as unobservable regions by the model (false positives), while blue indicates unobservable regions missed by the model (false negatives). The final row Difference Ratio shows the pixel-wise discrepancy between the model prediction and ground truth.