SeeLe: A Unified Acceleration Framework for Real-Time Gaussian Splatting

TL;DR

This work tackles the challenge of real-time Gaussian Splatting (3DGS) on resource-limited mobile devices by identifying three bottlenecks: computational intensity, rendering inefficiency, and memory budget. It introduces Seele, a unified framework with two GPU-oriented techniques: Hybrid Preprocessing (HP), which uses a view-dependent scene representation and offline clustering with online filtering to load only relevant Gaussians, and Contribution-Aware Rasterization (CR), which prioritizes high-contribution Gaussians and skips low-contribution ones to boost rasterization efficiency. An integrated fine-tuning step further preserves rendering quality while improving view consistency. Empirically, Seele achieves up to 6.3x speedups and substantial runtime model reductions (up to ~39%), with better rendering quality across multiple datasets and hardware configurations, demonstrating strong practical value for mobile real-time rendering and guiding future GPU-aware acceleration strategies for 3DGS.

Abstract

3D Gaussian Splatting (3DGS) has become a crucial rendering technique for many real-time applications. However, the limited hardware resources on today's mobile platforms hinder these applications, as they struggle to achieve real-time performance. In this paper, we propose SeeLe, a general framework designed to accelerate the 3DGS pipeline for resource-constrained mobile devices. Specifically, we propose two GPU-oriented techniques: hybrid preprocessing and contribution-aware rasterization. Hybrid preprocessing alleviates the GPU compute and memory pressure by reducing the number of irrelevant Gaussians during rendering. The key is to combine our view-dependent scene representation with online filtering. Meanwhile, contribution-aware rasterization improves the GPU utilization at the rasterization stage by prioritizing Gaussians with high contributions while reducing computations for those with low contributions. Both techniques can be seamlessly integrated into existing 3DGS pipelines with minimal fine-tuning. Collectively, our framework achieves 2.6$\times$ speedup and 32.3\% model reduction while achieving superior rendering quality compared to existing methods.

Figures (8)

• Figure 1: Our acceleration framework, Seele, achieves up to 6.3$\times$ speedup against the state-of-the-art 3DGS algorithms.
• Figure 2: The overview of Seele. we modify the two steps, preprocessing and rasterization, and propose two novel techniques: hybrid preprocessing and contribution-aware rasterization, in Gaussian splatting. Hybrid preprocessing leverages offline coarse-grained scene clustering and online filtering to reduce the number of Gaussians before rasterization. Contribution-aware rasterization dynamically identifies insignificant Gaussians and skips them to accelerate the overall rendering pipeline.
• Figure 3: Our scene representation clusters Gaussians into shared ones and exclusive ones. Here, we show the Gaussian positions without scales. The yellow points in Fig. \ref{['fig:cluster_w_shared']} represent the shared Gaussians, while the other colors correspond to the exclusive Gaussians in different clusters.
• Figure 4: The significance of Gaussians towards the final pixel. Gaussians are sorted in descending order. We empirically find that significant Gaussians are typically sampled in high-frequency regions, while insignificant Gaussians are more likely to be sampled in low-frequency regions (red crosses).
• Figure 5: An example of warp divergence in GPU. All threads compute $\alpha$ and then perform color blending in "lockstep". Our algorithm can detect the insignificant Gaussians (e.g., Gaussian $A$) and skip their color blending, as highlighted by the red cross. Thus, we save the total execution time.