Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Abstract

Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360° scene understanding for real-world applications. However, it remains particularly challenging due to severe geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360° panoramic images, explicitly tackling both geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.

Figures (11)

• Figure 1: Extrapolative Domain Adaptation (EDA) extends beyond the training FoV and known semantic categories, facilitating the transfer of knowledge from pinhole supervision to unlabeled 360° perception. It addresses both cross-view geometric distortions and the semantic extrapolation to unknown categories.
• Figure 2: Illustration of the proposed EDA-PSeg, which tackles cross-view and semantic extrapolation. The framework incorporates two key components: the Graph Matching Adapter (GMA), which constructs a high-order graph to align domain shared class graph nodes, and the Euler-Margin Attention (EMA), which models features with the Euler formula to enhance angle invariance under unseen viewpoints.
• Figure 3: (a) EulerFormer tian2024eulerformer fails to build a semantic angle constraint, limiting generalization to unseen views. (b) Our method employs Euler-Margin Projection to constrain the angle within the interval, and utilizes Amplitude and Phase Modulation to adjust the class distribution for known and unknown class separation.
• Figure 4: Feature distributions of the car category visualized via t-SNE across multiple datasets, illustrating the effects of Camera Types (Pinhole, Panoramic, Real, and Synthetic) and varying Weather Conditions (Fog, Rain, Cloudy, Sunny, Night, and Snow).
• Figure 5: T-SNE visualization of source and target domains in EDA-PSeg. (a) The prototype exhibits a relatively mixed unknown class. (b) Ours is the relatively separable unknown class.