Mìmir: A real-time interactive visualization library for CUDA programs

TL;DR

The paper addresses the challenge of real-time, in-situ visualization for GPU-based simulations by introducing Mìmir, a CUDA/Vulkan interoperability library. Mìmir provides a high-level API that manages interop-mapped memory, view descriptors, and runtime shader pipelines via Slang to render CUDA data with Vulkan, minimizing host-device data transfers and low-level graphics programming. The approach is demonstrated through multiple physics-inspired applications and a performance benchmark that highlights the importance of synchronization for balancing compute and graphics workloads. The results show that real-time visualization can be achieved with modest code changes, while synchronization mechanisms and Vulkan backends contribute to stability and scalability, with future work aimed at broader hardware support and remote rendering capabilities.

Abstract

Real-time visualization of computational simulations running over graphics processing units (GPU) is a valuable feature in modern science and technological research, as it allows researchers to visually assess the quality and correctness of their computational models during the simulation. Due to the high throughput involved in GPU-based simulations, classical visualization approaches such as ones based on copying to RAM or storage are not feasible anymore, as they imply large memory transfers between GPU and CPU at each moment, reducing both computational performance and interactivity. Implementing real-time visualizers for GPU simulation codes is a challenging task as it involves dealing with i) low-level integration of graphics APIs (e.g, OpenGL and Vulkan) into the general-purpose GPU code, ii) a careful and efficient handling of memory spaces and iii) finding a balance between rendering and computing as both need the GPU resources. In this work we present Mìmir, a CUDA/Vulkan interoperability C++ library that allows users to add real-time 2D/3D visualization to CUDA codes with low programming effort. With Mìmir, researchers can leverage state-of-the-art CUDA/Vulkan interoperability features without needing to invest time in learning the complex low-level technical aspects involved. Internally, Mìmir streamlines the interoperability mapping between CUDA device memory containing simulation data and Vulkan graphics resources, so that changes on the data are instantly reflected in the visualization. This abstraction scheme allows generating visualizations with minimal alteration over the original source code, needing only to replace the GPU memory allocation lines of the data to be visualized by the API calls provided by Mìmir among other optional changes.

Figures (15)

• Figure 1: Overview of Mìmir library processes. Only blue stages are handled by the user, while green stages are managed by the library itself.
• Figure 2: Overview of the view creation flow. Blue stages are handled by the library user, while green stages are managed by the library itself.
• Figure 3: Overview of the interop synchronization scheme. Blue stages are initiated by the user, while green stages are managed by the library.
• Figure 4: Point cloud visualization experiment visualized with Mìmir.
• Figure 5: Performance results for various combinations of framebuffer size and target FPS.