VR-Pipe: Streamlining Hardware Graphics Pipeline for Volume Rendering

TL;DR

The paper addresses the inefficiencies of rendering radiance-field models, specifically 3D Gaussian splatting, on fixed-function graphics hardware. It introduces VR-Pipe, a GPU pipeline augmentation with native early termination and quad merging to alleviate ROP bottlenecks and leverage shader-core blending for volume rendering. Through OpenGL-based experiments and Emerald/GPU hardware analysis, it demonstrates average speedups around 2.07× (up to 2.78× in some cases) and notable energy efficiency gains, while maintaining compatibility with standard graphics APIs. The work shows that modest hardware extensions—focused on a small hardware footprint—can substantially improve practical performance for volume rendering in real-time graphics pipelines.

Abstract

Graphics rendering that builds on machine learning and radiance fields is gaining significant attention due to its outstanding quality and speed in generating photorealistic images from novel viewpoints. However, prior work has primarily focused on evaluating its performance through software-based rendering on programmable shader cores, leaving its performance when exploiting fixed-function graphics units largely unexplored. In this paper, we investigate the performance implications of performing radiance field rendering on the hardware graphics pipeline. In doing so, we implement the state-of-the-art radiance field method, 3D Gaussian splatting, using graphics APIs and evaluate it across synthetic and real-world scenes on today's graphics hardware. Based on our analysis, we present VR-Pipe, which seamlessly integrates two innovations into graphics hardware to streamline the hardware pipeline for volume rendering, such as radiance field methods. First, we introduce native hardware support for early termination by repurposing existing special-purpose hardware in modern GPUs. Second, we propose multi-granular tile binning with quad merging, which opportunistically blends fragments in shader cores before passing them to fixed-function blending units. Our evaluation shows that VR-Pipe greatly improves rendering performance, achieving up to a 2.78x speedup over the conventional graphics pipeline with negligible hardware overhead.

Figures (23)

• Figure 1: Number of shader cores and render output units in the recent generations of top-of-the-line NVIDIA desktop GPUs. The labels on the bars indicate the absolute numbers for each.
• Figure 2: OpenGL rendering pipeline.
• Figure 3: NVIDIA Ampere GPU architecture ampere.
• Figure 4: Gaussian splatting rendering via graphics APIs.
• Figure 5: Performance comparison between software-based (CUDA) and hardware-based (OpenGL) graphics rendering on Jetson AGX Orin and RTX 3090. See Table \ref{['tab:workloads']} for the details of each scene.