SplatBus: A Gaussian Splatting Viewer Framework via GPU Interprocess Communication

TL;DR

The paper tackles the integration bottleneck of real-time Gaussian Splatting by decoupling the rasterizer from visualization through a server–client framework called SplatBus, enabling low-latency viewing in external apps via Nvidia IPC. The Python server exposes GPU frame buffers (color in RGBA32F and depth in R32F) and converts inverse depth $1/z$ to linear depth $z$, communicating with Unity, Blender, and OpenGL clients over two TCP channels. Clients implement their own rendering paths (e.g., a Unity plugin with OpenGL–CUDA interop for zero-copy texture transfers and an OpenGL viewer for debugging) while sharing a unified protocol, allowing hybrid rendering with meshes and dynamic scenes without modifying the core rasterizer. This approach reduces engineering effort, broadens the applicability of Gaussian Splatting to interactive and profiling workflows, and supports future extensions like shadows, lighting, and cross-platform deployment.

Abstract

Radiance field-based rendering methods have attracted significant interest from the computer vision and computer graphics communities. They enable high-fidelity rendering with complex real-world lighting effects, but at the cost of high rendering time. 3D Gaussian Splatting solves this issue with a rasterisation-based approach for real-time rendering, enabling applications such as autonomous driving, robotics, virtual reality, and extended reality. However, current 3DGS implementations are difficult to integrate into traditional mesh-based rendering pipelines, which is a common use case for interactive applications and artistic exploration. To address this limitation, this software solution uses Nvidia's interprocess communication (IPC) APIs to easily integrate into implementations and allow the results to be viewed in external clients such as Unity, Blender, Unreal Engine, and OpenGL viewers. The code is available at https://github.com/RockyXu66/splatbus.

Figures (3)

• Figure 1: System diagram of our solution. Our server can be easily integrated into any codebase as a set of Python API calls. The camera controls and other arbitrary data are transferred to and from the client (e.g. Unity, an OpenGL application) via sockets, while the rendered frame is transferred via Nvidia's Inter-Process Communication (IPC).
• Figure 2: Unity plugin visualizing real-time 3D Gaussian Splatting results with interactive point clouds and 3D meshes.
• Figure 3: Unity plugin visualizing real-time Gaussian avatar 11094909 with 3D meshes.