BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis

TL;DR

BillBoard Splatting (BBSplat) tackles the efficiency-accuracy gap in novel view synthesis by representing scenes with learnable textured planar primitives (billboards) that replace 3D Gaussians in Gaussian Splatting pipelines. By equipping each billboard with an RGB texture and an alpha map, BBSplat permits arbitrary shapes, high-frequency detail, and accurate mesh extraction, while enabling ray-tracing-like rasterization effects. The approach introduces a texture-based regularization and a compression pipeline that yields substantial storage reductions (up to ~7x on average and up to x17x vs 3DGS) without sacrificing rendering quality, achieving state-of-the-art PSNR on DTU and competitive results on Tanks&Temples and Mip-NeRF-360. Overall, BBSplat provides a scalable, photorealistic NVS framework with explicit geometry suitable for mesh extraction and rendering, improving practicality for real-world applications and downstream tasks.

Abstract

We present billboard Splatting (BBSplat) - a novel approach for novel view synthesis based on textured geometric primitives. BBSplat represents the scene as a set of optimizable textured planar primitives with learnable RGB textures and alpha-maps to control their shape. BBSplat primitives can be used in any Gaussian Splatting pipeline as drop-in replacements for Gaussians. The proposed primitives close the rendering quality gap between 2D and 3D Gaussian Splatting (GS), enabling the accurate extraction of 3D mesh as in the 2DGS framework. Additionally, the explicit nature of planar primitives enables the use of the ray-tracing effects in rasterization. Our novel regularization term encourages textures to have a sparser structure, enabling an efficient compression that leads to a reduction in the storage space of the model up to x17 times compared to 3DGS. Our experiments show the efficiency of BBSplat on standard datasets of real indoor and outdoor scenes such as Tanks&Temples, DTU, and Mip-NeRF-360. Namely, we achieve a state-of-the-art PSNR of 29.72 for DTU at Full HD resolution.

Figures (8)

• Figure 1: Method description. a) As input we use point cloud and camera positions predicted with COLMAP schoenberger2016sfm. b) Our BBSplat parametrization (b2) extends Gaussian primitives parametrization (b1) with two textures for each point: RGB texture for colors and alpha texture for transparency. c) As defined in 2DGS splatting technique (c1), we find the ray-plane intersection, but instead of calculating Gaussian opacity, we sample color and opacity from the texture (c2). d) To train our 3D scene representation, we utilize only photometric losses.
• Figure 2: Texture rasterization. During rendering to get final billboard colors we add texture $T^\textrm{RGB}_i$ colors to base color calculated with $\textrm{SH}_i$ and view-direction vector $\textbf{d}_i$. It results in textured billboards handling light effects. Here we showcase four possible renderings for different camera directions.
• Figure 3: Qualitative results. We provide rendering results of scenes from each dataset: Tanks&Temples [Train], Mip-NeRF-360 [Room], and DTU [Scan97]. For competitors methods, we use the maximum number of Gaussians recommended by the method. More renderings can be found in the \ref{['sec:results_images']} of Supp. Mat..
• Figure 4: Ray-tracing and mesh extraction. a) BBSplat representation of the scene as explicit textured planes allows the implementation of ray-tracing effects during rasterization. b) Rasterization of planes with alpha texture provide accurate objects suitable for mesh extraction. c) Alternatively, our representation can be used to extract accurate 3D meshes, surpassing the accuracy of 3D Gaussians-based methods (e.g. SuGaR). More renderings and extracted meshes can be found in the \ref{['sec:results_images']} of Supp. Mat..
• Figure 5: Sky-sphere around the scene. We sample additional points on the sphere around all SfM points with the Fibonacci algorithm gonzalez2010measurement.