TOGS: Gaussian Splatting with Temporal Opacity Offset for Real-Time 4D DSA Rendering

TL;DR

TOGS addresses real-time, high-quality 4D DSA novel-view synthesis under sparse views by extending 3D Gaussian Splatting with per-Gaussian opacity offsets over time. Each Gaussian carries an initial opacity $\alpha'$ and a time-dependent offset $\Delta\alpha_t$ stored in an opacity-offset table, yielding $\alpha_t = \alpha' + \Delta\alpha_t$, and rendering proceeds via differentiable Gaussian rasterization. A Smooth loss plus random Gaussian pruning mitigates overfitting and reduces storage, enabling fast training on standard hardware. Evaluations on TiAVox patient data show state-of-the-art performance at sparse viewpoints and real-time rendering (over 300 FPS) with per-patient Gaussian models under ~20 MB, suggesting substantial clinical impact in reducing radiation exposure and enabling interactive visualization. Overall, TOGS provides a practical, interpretable, and efficient approach to 4D DSA rendering that bridges the gap between speed, quality, and storage efficiency in medical imaging.

Abstract

Four-dimensional Digital Subtraction Angiography (4D DSA) is a medical imaging technique that provides a series of 2D images captured at different stages and angles during the process of contrast agent filling blood vessels. It plays a significant role in the diagnosis of cerebrovascular diseases. Improving the rendering quality and speed under sparse sampling is important for observing the status and location of lesions. The current methods exhibit inadequate rendering quality in sparse views and suffer from slow rendering speed. To overcome these limitations, we propose TOGS, a Gaussian splatting method with opacity offset over time, which can effectively improve the rendering quality and speed of 4D DSA. We introduce an opacity offset table for each Gaussian to model the opacity offsets of the Gaussian, using these opacity-varying Gaussians to model the temporal variations in the radiance of the contrast agent. By interpolating the opacity offset table, the opacity variation of the Gaussian at different time points can be determined. This enables us to render the 2D DSA image at that specific moment. Additionally, we introduced a Smooth loss term in the loss function to mitigate overfitting issues that may arise in the model when dealing with sparse view scenarios. During the training phase, we randomly prune Gaussians, thereby reducing the storage overhead of the model. The experimental results demonstrate that compared to previous methods, this model achieves state-of-the-art render quality under the same number of training views. Additionally, it enables real-time rendering while maintaining low storage overhead. The code is available at https://github.com/hustvl/TOGS.

Figures (8)

• Figure 1: Comparison of temporal variations between natural scenes and 4D DSA scenes.
• Figure 3: An overview of the proposed TOGS for 4D DSA novel view synthesis. DSA data (a) is collected from different viewpoints and time points. The image is then denoised to retain only the blood vessel region. The denoised data (b) is subsequently used to train the TOGS model. The trained TOGS model is capable of rendering DSA images (c) from any desired time point and viewpoint.
• Figure 4: Overview of our pipeline. First, we randomly generate point clouds and initialize them as 3D Gaussians. We introduce an opacity offset table for each Gaussian. When different time values $t$ are input, interpolate the opacity offset table to obtain the opacity offset value $\Delta \alpha_{t}$. Add it to the original opacity $\alpha^{\prime}$ to obtain the opacity $\alpha_{t}$ at that moment. Then, Gaussians are input into the efficient differential Gaussian rasterization pipeline to render the image. Finally, by computing the loss and backpropagating gradients, we adaptively control the density of the Gaussians.
• Figure 5: Visualization comparison results of different methods in the 4D DSA new view synthesis task. (Number of training and testing views: 30/103, TR: Temporal resolution).
• Figure 6: The impact of randomly pruning Gaussians at different proportions on the number of Gaussians and the quality of generated images (30 training views).