In-SRAM Radiant Foam Rendering on a Graph Processor

TL;DR

This work tackles volumetric radiance-field rendering on distributed-memory accelerators by keeping both scene data and ray state entirely in tile-local SRAM. It introduces an end-to-end Radiant Foam renderer on the Graphcore IPU that partitions Voronoi cells across tiles and routes rays through a static hierarchical quadtree, avoiding global memory accesses. Key contributions include a kd-tree-based scene partitioning into 1024 shards, a compact per-ray payload, a two-pass router scheme for deterministic routing, and screen-space framebuffer distribution, enabling on-chip rendering with near-interactive throughput (~1 fps at 640×480) on Mip-NeRF 360 and Deep Blending scenes. The paper also analyzes routing, memory footprint, and scheduling bottlenecks to inform design choices for future distributed-memory accelerators in irregular, data-mmovement-heavy rendering workloads, highlighting both feasibility and practical limitations such as root-level routing bottlenecks and camera-pose desynchronization risks.

Abstract

Many emerging many-core accelerators replace a single large device memory with hundreds to thousands of lightweight cores, each owning only a small local SRAM and exchanging data via explicit on-chip communication. This organization offers high aggregate bandwidth, but it breaks a key assumption behind many volumetric rendering techniques: that rays can randomly access a large, unified scene representation. Rendering efficiently on such hardware therefore requires distributing both data and computation, keeping ray traversal mostly local, and structuring communication into predictable routes. We present a fully in-SRAM, distributed renderer for the \emph{Radiant Foam} Voronoi-cell volumetric representation on the Graphcore Mk2 IPU, a many-core accelerator with tile-local SRAM and explicit inter-tile communication. Our system shards the scene across tiles and forwards rays between shards through a hierarchical routing overlay, enabling ray marching entirely from on-chip SRAM with predictable communication. On Mip-NeRF~360 scenes, the system attains near-interactive throughput (\(\approx\)1\,fps at \mbox{$640\times480$}) with image and depth quality close to the original GPU-based Radiant Foam implementation, while keeping all scene data and ray state in on-chip SRAM. Beyond demonstrating feasibility, we analyze routing, memory, and scheduling bottlenecks that inform how future distributed-memory accelerators can better support irregular, data-movement-heavy rendering workloads.

Figures (26)

• Figure 1: System overview. A trained Radiant Foam scene is partitioned into SRAM-sized shards and placed on IPU tiles acting as ray tracers. During rendering, rays are forwarded through a sparse quadtree routing overlay (illustrated with three router levels for clarity) when they cross shard boundaries, transferring ray state to the tile that owns the next shard. The output image is also partitioned across tiles; each ray’s final pixel value is routed to the tile responsible for that pixel region.
• Figure 2: Intersection in Radiant Foam (2D illustration). Primal points are shown as dots and labelled $p_c$ for the current cell and $p_{n1}, p_{n2}, p_{n3}$ for neighboring cells. Their Voronoi cells appear as polygonal regions separated by straight line segments. A single ray (red arrow) traverses multiple neighboring cells: within the current cell, it intersects several candidate faces, with the corresponding distances $t_1, t_2, t_3$ calculated. The ray then crosses the face with the smallest positive distance (here $t_2$) into the adjacent cell associated with $p_{n2}$.
• Figure 3: Partition of the Voronoi diagram. Local cells in a partition require access to the positions of neighbor cells across boundaries in order to reconstruct their Voronoi cell shapes during rendering.
• Figure 4: Quadtree-based routing overlay(routing tree). Only rightmost branches are shown for clarity.
• Figure 5: Operation of a ray router tile. Rays are received from the parent and four children and forwarded to the correct output lane based on their target partition.