XYZCylinder: Towards Compatible Feed-Forward 3D Gaussian Splatting for Driving Scenes via Unified Cylinder Lifting Method

TL;DR

XYZCylinder presents a foundation for compatible feed-forward 3D Gaussian splatting of driving scenes by introducing a Unified Cylinder Lifting method. The core ideas are Unified Cylinder Camera Modeling (UCCM) for explicit, training-free viewpoint handling and Cylinder Plane Feature Groups (CPFG) for a hybrid foreground/background representation, enabling zero-shot generalization across diverse camera configurations. The method decouples foreground and background, employs occupancy-, volume-, and pixel-aware modules to generate 3D Gaussians, and fuses a high-fidelity background via a StyleGAN-based 2D-to-3D pipeline. Across nuScenes and Carla-Centric, XYZCylinder achieves state-of-the-art reconstruction quality and demonstrates strong cross-dataset compatibility, including zero-shot transfer to unseen camera setups and datasets, highlighting its practical potential for autonomous driving simulation and perception augmentation.

Abstract

Feed-forward paradigms for 3D reconstruction have become a focus of recent research, which learn implicit, fixed view transformations to generate a single scene representation. However, their application to complex driving scenes reveals significant limitations. Two core challenges are responsible for this performance gap. First, the reliance on a fixed view transformation hinders compatibility to varying camera configurations. Second, the inherent difficulty of learning complex driving scenes from sparse 360° views with minimal overlap compromises the final reconstruction fidelity. To handle these difficulties, we introduce XYZCylinder, a novel method built upon a unified cylinder lifting method that integrates camera modeling and feature lifting. To tackle the compatibility problem, we design a Unified Cylinder Camera Modeling (UCCM) strategy. This strategy explicitly models projection parameters to unify diverse camera setups, thus bypassing the need for learning viewpoint-dependent correspondences. To improve the reconstruction accuracy, we propose a hybrid representation with several dedicated modules based on newly designed Cylinder Plane Feature Group (CPFG) to lift 2D image features to 3D space. Extensive evaluations confirm that XYZCylinder not only achieves state-of-the-art performance under different evaluation settings but also demonstrates remarkable compatibility in entirely new scenes with different camera settings in a zero-shot manner. Project page: \href{https://yuyuyu223.github.io/XYZCYlinder-projectpage/}{here}

Figures (30)

• Figure 1: Reconstruction results of the proposed method under different evaluation settings. Our model achieves better reconstruction quality and better compatibility.
• Figure 2: Overview of XYZCylinder. The scene is reconstructed in three stages with the unified cylinder camera modeling for feature extraction and a hybrid representation with different dedicated modules for foreground and background reconstruction.
• Figure 3: Overview of the unified cylinder camera modeling (UCCM). The design of UCCM empowers our model with zero-shot generalization across different datasets.
• Figure 4: (a) Architecture of Y-shaped network for the occupancy-aware module YNet$_{\rm occ}$ and pixel-aware module YNet$_{\rm pix}$. The network is mainly implemented based on the ResNet Block he2016deep and EMA EMAttention. (b) The cylinder plane feature group is constructed by splitting the feature in the channel dimension.
• Figure 5: (a) Architecture of XNet$_{\rm vol}$, which is composed of a dual-branch encoder for downsampling and a dual-branch decoder for appearance and geometric feature upsampling. (b) Overview of the Z-Net architecture. It includes upsampling, ray casting and synthesis.

Theorems & Definitions (1)

• Definition 1