360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception

TL;DR

This work tackles 360 panoramic room layout estimation under distortion and data-label constraints. It introduces DOPNet to disentangle orthogonal planes and produce distortion-free horizon-depth $d$ and height ratio $R$ representations, and extends to MV-DOPNet for unsupervised multi-view adaptation via decision-level (D2L/L2D) optimization and feature-level 1D cost volumes. Key contributions include distortion-aware feature aggregation, soft-flipping fusion for symmetry cues, a sequence reconstruction module with discriminative channels and long-range dependencies, and a ceiling-3D constrained loss to align horizon-depth with ceiling-depth. The approach achieves state-of-the-art results on monocular and multi-view layout tasks across cuboid and general room datasets, demonstrating improved robustness to distortions, occlusions, and domain shifts while reducing reliance on labeled data. The practical impact lies in more accurate, data-efficient 360 room layout understanding applicable to AR/VR, real estate, and robotics.

Abstract

Existing panoramic layout estimation solutions tend to recover room boundaries from a vertically compressed sequence, yielding imprecise results as the compression process often muddles the semantics between various planes. Besides, these data-driven approaches impose an urgent demand for massive data annotations, which are laborious and time-consuming. For the first problem, we propose an orthogonal plane disentanglement network (termed DOPNet) to distinguish ambiguous semantics. DOPNet consists of three modules that are integrated to deliver distortion-free, semantics-clean, and detail-sharp disentangled representations, which benefit the subsequent layout recovery. For the second problem, we present an unsupervised adaptation technique tailored for horizon-depth and ratio representations. Concretely, we introduce an optimization strategy for decision-level layout analysis and a 1D cost volume construction method for feature-level multi-view aggregation, both of which are designed to fully exploit the geometric consistency across multiple perspectives. The optimizer provides a reliable set of pseudo-labels for network training, while the 1D cost volume enriches each view with comprehensive scene information derived from other perspectives. Extensive experiments demonstrate that our solution outperforms other SoTA models on both monocular layout estimation and multi-view layout estimation tasks. Cobe can be available at https://github.com/zhijieshen-bjtu/MV-DOPNet.

Figures (9)

• Figure 1: (a) The commonly used architecture. (b) DOPNet. (c) MV-DOPNet. Compared with the traditional pipeline, ours has the following advantages: (1) Disentangling the orthogonal planes to get distortion-free, and semantic-clean, detail-sharp representations. (2) Leveraging multi-view geometric consistency at the feature level (MV-GCP module). (3) Optimization strategy suitable for horizon-depth/ratio layout representations.
• Figure 2: Workflow of the proposed unsupervised multi-view layout framework. Our approach adopts panoramic images as input and predicts the corresponding horizon-depth map and the height (ratio). Then they are employed to recover the layout boundaries via D2L transformation. We leverage the multi-view layout consistency to optimize the boundaries and convert them back to the original format via L2D transformation. Finally, the optimized pseudo-labels are employed to fine-tune the MV-DOPNet (extending DOPNet by designing a multi-view geometric consistency perceptron module).
• Figure 3: MV-DOPNet. The feature assembling mechanism is designed to deal with distortions as well as integrate shallow and deep features. Then we pre-segment orthogonal planes to produce two 1D representations with distinguished plane semantics. A sequence reconstruction module is deployed to reconstruct the 1D representations. We design multi-view geometric consistency perceptron to incorporate multi-view insights. Our scheme leverages a sequence of panoramas with a resolution of 512×1024 as input and predicts the horizon-depth and the associated ratio value.
• Figure 4: Illustration of the misaligned case when leveraging the symmetry property.
• Figure 5: Qualitative comparison results evaluated on cuboid layout datasets, Stanford 2D-3D armeni2017joint and PanoContext zhang2014panocontext. The boundaries of the room layout on a panorama are shown on the left and the floor plan is on the right. Ground truth is best viewed in blue lines and the prediction in green. The predicted horizon depth, normal, and gradient are visualized below each panorama, and the ground truth is in the first row.