3D Gabor Splatting: Reconstruction of High-frequency Surface Texture using Gabor Noise

TL;DR

Addresses the challenge of reconstructing high-frequency surface texture with differentiable 3D radiance representations derived from Gaussian splatting. Proposes 3D Gabor splatting where each primitive uses a kernel composed of $N=4$ wave functions with learnable frequencies $f_i$ and fixed uniformly sampled orientations, enabling richer spatial textures without a heavy increase in primitive count. The color is blended by $\mathbf{c}(u,v)=\sum_{i=0}^{N-1} w_i\left(\mathbf{c}_A\frac{1+\cos\theta_i}{2}+\mathbf{c}_B\frac{1-\cos\theta_i}{2}\right)$ with $\theta_i(u,v)=2\pi f_i\left(\cos\frac{i\pi}{N},\sin\frac{i\pi}{N}\right)(u,v)^T+\phi_i$, and optimization runs for $30{,}000$ iterations initializing kernel centers from Structure-from-Motion. Empirical results on four garment datasets show improved SSIM, PSNR, and LPIPS relative to $2$DGS, with faster convergence and rendering speeds competitive with real-time rates. The work demonstrates practical improvements for high-frequency texture reconstruction in view-synthesis pipelines and outlines future directions, including view-dependent color with spherical harmonics and CUDA-based acceleration.

Abstract

3D Gaussian splatting has experienced explosive popularity in the past few years in the field of novel view synthesis. The lightweight and differentiable representation of the radiance field using the Gaussian enables rapid and high-quality reconstruction and fast rendering. However, reconstructing objects with high-frequency surface textures (e.g., fine stripes) requires many skinny Gaussian kernels because each Gaussian represents only one color if viewed from one direction. Thus, reconstructing the stripes pattern, for example, requires Gaussians for at least the number of stripes. We present 3D Gabor splatting, which augments the Gaussian kernel to represent spatially high-frequency signals using Gabor noise. The Gabor kernel is a combination of a Gaussian term and spatially fluctuating wave functions, making it suitable for representing spatial high-frequency texture. We demonstrate that our 3D Gabor splatting can reconstruct various high-frequency textures on the objects.

Figures (3)

• Figure 2: Construction of our Gabor kernel. The multiple wave functions, whose directions are uniformly sampled, are composed in the local coordinate of the kernel. Then, affine transformation computes the color distribution in the object space.
• Figure 3: Qualitative and quantitative ablation study against three baselines. The single wave function inside a primitive (Baseline A) shows blurry output. The multiple wave functions in a single wave orientation (Baseline B) show false color. The effect of removing phase shift (Baseline C) is small in appearance, but the quantitative scores (right) slightly degrade. The Ours* is our result with approximately half the number of primitives, reducing the data size to the same as the 2DGS (both 2.411MB), while still showing improvements against the 2DGS.
• Figure 4: Comparison of the converged results using the 2D Gaussian splatting (2DGS) huang20242d (left) and our (middle) model. Our model converges faster to a smaller loss (right). The Shirt (top) dataset has 4.5k primitives, and the Boots (bottom) dataset has 4.2k primitives.