GStex: Per-Primitive Texturing of 2D Gaussian Splatting for Decoupled Appearance and Geometry Modeling

TL;DR

GStex addresses the coupling of appearance and geometry in Gaussian splatting by introducing per-Gaussian texture maps that independently encode albedo while retaining Gaussian geometry. It initializes from standard 2D Gaussian splatting and jointly optimizes textures with view-dependent components through a two-stage process, enabling fine-grained appearance edits and robust novel-view synthesis with fewer primitives. The method combines texture lookups with spherical harmonics in a rendering model and supports texture painting and procedural re-texturing, yielding improved high-frequency detail and editing capabilities compared to prior texture approaches. GStex thus bridges Gaussian-based scene representations with conventional texturing paradigms, offering enhanced visual fidelity, editability, and potential integration with traditional graphics pipelines.

Abstract

Gaussian splatting has demonstrated excellent performance for view synthesis and scene reconstruction. The representation achieves photorealistic quality by optimizing the position, scale, color, and opacity of thousands to millions of 2D or 3D Gaussian primitives within a scene. However, since each Gaussian primitive encodes both appearance and geometry, these attributes are strongly coupled--thus, high-fidelity appearance modeling requires a large number of Gaussian primitives, even when the scene geometry is simple (e.g., for a textured planar surface). We propose to texture each 2D Gaussian primitive so that even a single Gaussian can be used to capture appearance details. By employing per-primitive texturing, our appearance representation is agnostic to the topology and complexity of the scene's geometry. We show that our approach, GStex, yields improved visual quality over prior work in texturing Gaussian splats. Furthermore, we demonstrate that our decoupling enables improved novel view synthesis performance compared to 2D Gaussian splatting when reducing the number of Gaussian primitives, and that GStex can be used for scene appearance editing and re-texturing.

Figures (13)

• Figure 1: GStex. We apply 2D Gaussian splatting (2DGS) kerbl20233d and GStex to reconstruct a planar textured scene (a whiteboard; left column) and edit its appearance. (Col. 2) In 2DGS, each 2D Gaussian encodes both appearance (blue) and geometry (green), and so these properties are coupled. Our approach decouples appearance and geometry through use of per-Gaussian texture maps. Each tile of the blue checkerboard pattern represents $10 \times 10$ texels. For visualization purposes, we show Gaussians with opacity $>0.5$. (Cols. 3--4) When rendering the edited scene from novel views, our representation results in sharper details because it models features that are smaller in size than a single Gaussian. (Cols. 5--7) We find that GStex renders edited textures more accurately than 2DGS, even when using 200,000 Gaussians.
• Figure 2: Overview of the GStex rendering model. We initialize the model using 2DGS huang20242d and assign an RGB texture map $\mathbf{c}^{\mathcal{T}}_i$ to each 2D Gaussian primitive $G^\mathcal{T}_i$. We render a pixel by casting a ray into the scene; the ray passes through a textured 2D Gaussian that is oriented along $\mathbf{v}_{\mathbf{1}i}$ and $\mathbf{v}_{\mathbf{2}i}$ with position $\boldsymbol{\mu}_i$ and normal $\mathbf{n}_i$. The color $\mathbf{c}_i$ is given by mapping the intersection point to the uv coordinates of the Gaussian's texture map $\mathbf{c}^{\mathcal{T}}_i$, bilinearly interpolating the resulting texture value, and adding it to the Gaussian's view-dependent color component $\mathbf{c}^{\text{SH}}_i$ (parameterized by spherical harmonics). The alpha value $\alpha_i$ is given by evaluating the Gaussian function at the intersection point. Finally, we alpha composite the rendered colors of each intersected Gaussian, resulting in the rendered pixel color $\hat{\mathbf{c}}$.
• Figure 3: Comparison to Texture-GS. We compare renders of our method and Texture-GS xu2024texture on scenes from the Blender dataset, which generally have finer geometry compared to the DTU dataset. High-frequency scene textures are rendered more clearly using GStex compared to Texture-GS.
• Figure 4: Novel view synthesis for varying numbers of Gaussians. We perform novel view synthesis using our representation (solid) and 2DGS (dashed) initialized with varying numbers of Gaussians ranging from $128$ to $32768$. Densification and culling are turned off to ensure a consistent number of Gaussians. The plots show metrics averaged across all Blender synthetic scenes (blue) and DTU scenes (red).
• Figure 5: Ablation over number of texels and Gaussians. We evaluate the effect of the number of texels as the number of Gaussians varies. Densification and culling are turned off to ensure a consistent number of Gaussians. This experiment is performed on the whiteboard scene (Figure \ref{['fig:teaser']}).