Layout Anything: One Transformer for Universal Room Layout Estimation

TL;DR

Layout Anything introduces a transformer-based, end-to-end framework for indoor room layout estimation that adapts the OneFormer segmentation backbone to predict geometrically consistent planar surfaces. It fuses task-conditioned segmentation with geometry-aware regularization and a topology-preserving degeneration strategy to improve generalization without post-processing. The method achieves state-of-the-art performance on LSUN, Hedau, and Matterport3D-Layout, with fast inference suitable for AR and 3D reconstruction tasks. Overall, the approach demonstrates strong accuracy, speed, and robustness across both cuboid and non-cuboid room layouts.

Abstract

We present Layout Anything, a transformer-based framework for indoor layout estimation that adapts the OneFormer's universal segmentation architecture to geometric structure prediction. Our approach integrates OneFormer's task-conditioned queries and contrastive learning with two key modules: (1) a layout degeneration strategy that augments training data while preserving Manhattan-world constraints through topology-aware transformations, and (2) differentiable geometric losses that directly enforce planar consistency and sharp boundary predictions during training. By unifying these components in an end-to-end framework, the model eliminates complex post-processing pipelines while achieving high-speed inference at 114ms. Extensive experiments demonstrate state-of-the-art performance across standard benchmarks, with pixel error (PE) of 5.43% and corner error (CE) of 4.02% on the LSUN, PE of 7.04% (CE 5.17%) on the Hedau and PE of 4.03% (CE 3.15%) on the Matterport3D-Layout datasets. The framework's combination of geometric awareness and computational efficiency makes it particularly suitable for augmented reality applications and large-scale 3D scene reconstruction tasks.

Figures (6)

• Figure 1: Our end-to-end proposed method pipeline.
• Figure 2: Our Layout Anything Framework Architecture. (a) Multi-scale feature extraction using a DiNAT backbone and pixel decoder. (b) Task-conditioned query formulation with transformer-guided queries, task token ($\mathbf{Q}_{\text{task}}$), and contrastive learning. (c) Multi-stage transformer decoder producing task-dynamic class and mask predictions. Note: The input mask is only used during the training phase for applying the degeneration strategy. It is not required at test time.
• Figure 3: Adaptive edge maps. (Top) Ground truth edge map generation. (Bottom) Edge maps with adaptive edge constraint applied.
• Figure 4: The composition of room types within the LSUN Room Layout dataset.
• Figure 5: Predicted results from our Layout Anything model.