Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rendering Large Volume Datasets in Unreal Engine 5: A Survey

Markus Schlüter, Tom Kwasnitschka, Armin Bernstetter, Jens Karstens

TL;DR

The paper surveys in-core volume rendering of large volumetric datasets in Unreal Engine 5 within the ARENA2 dome, comparing a custom approach, the TBRayMarcher plugin, Niagara Fluids, and Sparse Volume Textures with Heterogeneous Volumes. It identifies SVT/HV as the most promising path, while detailing practical limits due to VRAM, API, and 32-bit/64-bit arithmetic constraints, and proposing fixes to enable datasets approaching several gigavoxels. A key contribution is the identification and repair of int32 overflow issues that unlock near the theoretical SVT capacity of $2048^3$ voxel textures, though true out-of-core rendering remains essential for truly massive data. The work also discusses alternatives (ParaView/IndeX, scenery) and outlines a clear roadmap toward scalable, out-of-core volume rendering in UE5 for scientific visualization in dome-based setups.

Abstract

In this technical report, we discuss several approaches to in-core rendering of large volumetric datasets in Unreal Engine 5 (UE5). We explore the following methods: the TBRayMarcher Plugin, the Niagara Fluids Plugin , and various approaches using Sparse Volume Textures (SVT), with a particular focus on Heterogeneous Volumes (HV). We found the HV approach to be the most promising. The biggest challenge we encountered with other approaches was the need to chunk datasets so that each fits into volume textures smaller than one gigavoxel. While this enables display of the entire dataset at reasonable frame rates, it introduces noticeable artifacts at chunk borders due to incorrect lighting, as each chunk lacks information about its neighbors. After addressing some (signed) int32 overflows in the Engine's SVT-related source code by converting them to to (unsigned) uint32 or int64, the SVT-based HV system allows us to render sparse datasets up to 32k x 32k x 16k voxels, provided the compressed tile data (including MIP data and padding for correct interpolation) does not exceed 4 gigavoxels. In the future, we intend to extend the existing SVT streaming functionality to support out-of-core rendering, in order to eventually overcome VRAM limitations, graphics API constraints, and the performance issues associated with 64-bit arithmetic in GPU shaders.

Rendering Large Volume Datasets in Unreal Engine 5: A Survey

TL;DR

The paper surveys in-core volume rendering of large volumetric datasets in Unreal Engine 5 within the ARENA2 dome, comparing a custom approach, the TBRayMarcher plugin, Niagara Fluids, and Sparse Volume Textures with Heterogeneous Volumes. It identifies SVT/HV as the most promising path, while detailing practical limits due to VRAM, API, and 32-bit/64-bit arithmetic constraints, and proposing fixes to enable datasets approaching several gigavoxels. A key contribution is the identification and repair of int32 overflow issues that unlock near the theoretical SVT capacity of voxel textures, though true out-of-core rendering remains essential for truly massive data. The work also discusses alternatives (ParaView/IndeX, scenery) and outlines a clear roadmap toward scalable, out-of-core volume rendering in UE5 for scientific visualization in dome-based setups.

Abstract

In this technical report, we discuss several approaches to in-core rendering of large volumetric datasets in Unreal Engine 5 (UE5). We explore the following methods: the TBRayMarcher Plugin, the Niagara Fluids Plugin , and various approaches using Sparse Volume Textures (SVT), with a particular focus on Heterogeneous Volumes (HV). We found the HV approach to be the most promising. The biggest challenge we encountered with other approaches was the need to chunk datasets so that each fits into volume textures smaller than one gigavoxel. While this enables display of the entire dataset at reasonable frame rates, it introduces noticeable artifacts at chunk borders due to incorrect lighting, as each chunk lacks information about its neighbors. After addressing some (signed) int32 overflows in the Engine's SVT-related source code by converting them to to (unsigned) uint32 or int64, the SVT-based HV system allows us to render sparse datasets up to 32k x 32k x 16k voxels, provided the compressed tile data (including MIP data and padding for correct interpolation) does not exceed 4 gigavoxels. In the future, we intend to extend the existing SVT streaming functionality to support out-of-core rendering, in order to eventually overcome VRAM limitations, graphics API constraints, and the performance issues associated with 64-bit arithmetic in GPU shaders.

Paper Structure

This paper contains 22 sections, 1 equation, 18 figures, 1 table.

Table of Contents

  1. Introduction
  2. Requirements
  3. Background and Related Work
  4. Hardware Setup
  5. The Different Approaches
  6. Custom Implementation
  7. Using an existing Plugin: TBRayMarcher
  8. Adapting an existing System: Niagara Fluids
  9. Using an experimental feature: Sparse Volume Textures (SVT) / Heterogeneous Volumes
  10. Introduction to SVT
  11. Different Ways to render SVTs
  12. Importing SVT via OpenVDB
  13. Workaround: Create a 4-in-1 Material
  14. Increasing the Engine's limits
  15. Fixing the VDB import
  16. ...and 7 more sections

Figures (18)

  • Figure 1: The Kolumbo dataset rendered as a Heterogeneous Volume in a custom build of Unreal Engine 5.4, achieving a frame rate of 20 fps on an NVIDIA RTX 3500 Ada Generation Laptop GPU. The total (dense) volume resolution is $4211 \times 1501 \times 935 \approx 6$ billion voxels, while it has about 2.6 billion non-empty voxels.
  • Figure 2: The ARENA2 visualization dome at the GEOMAR Helmholtz Center for Ocean Research Kiel
  • Figure 3: "Painting" a volume in our ARENA2 visualization dome.
  • Figure 4: Illustration of secondary shadow ray marches. Source: bib:shaderBits2016
  • Figure 5: Illustration an illumination cache volume for one yellow and one blue point light source illuminating a group of spheres. Source: bib:TBRM_basePaper_Sunden
  • ...and 13 more figures