HoRAMA: Holistic Reconstruction with Automated Material Assignment for Ray Tracing using NYURay

TL;DR

HoRAMA addresses the bottleneck of creating site-specific RT environment models by automating RT-compatible 3D reconstruction from RGB video. It combines MASt3R-SLAM for dense point clouds, PTv3 for semantic segmentation, and Qwen3-VL for material labeling, with PLANA3R and Poisson-based surface reconstruction to produce Mitsuba XML meshes with ITU-R P.2040 material properties. Dual-band validation in a factory environment shows HoRAMA achieves RMSEs of 2.28 dB, closely matching the manual baseline of 2.18 dB while reducing reconstruction time from two months to 16 hours. This enables scalable wireless digital twins for RT-based network planning, beam management, and edge deployment in 5G/6G, with potential expansion to outdoor scenarios using drone-based video capture.

Abstract

Next-generation wireless networks at upper mid-band and millimeter-wave frequencies require accurate site-specific deterministic channel propagation prediction. Wireless ray tracing (RT) provides site-specific predictions but demands high-fidelity three-dimensional (3D) environment models with material properties. Manual 3D model reconstruction achieves high accuracy but requires weeks of expert effort, creating scalability bottlenecks for large environment reconstruction. Traditional vision-based 3D reconstruction methods lack RT compatibility due to geometrically defective meshes and missing material properties. This paper presents Holistic Reconstruction with Automated Material Assignment (HoRAMA) for wireless propagation prediction using NYURay. HoRAMA generates RT-compatible 3D models from RGB video readily captured using a smartphone or low-cost portable camera, by integrating MASt3R-SLAM dense point cloud generation with vision language model-assisted material assignment. The HoRAMA 3D reconstruction method is verified by comparing NYURay RT predictions, using both manually created and HoRAMA-generated 3D models, against field measurements at 6.75 GHz and 16.95 GHz across 12 TX-RX locations in a 700 square meter factory. HoRAMA ray tracing predictions achieve a 2.28 dB RMSE for matched multipath component (MPC) power predictions, comparable to the manually created 3D model baseline (2.18 dB), while reducing 3D reconstruction time from two months to 16 hours. HoRAMA enables scalable wireless digital twin creation for RT network planning, infrastructure deployment, and beam management in 5G/6G systems, as well as eventual real-time implementation at the edge.

Figures (4)

• Figure 1: Ray tracing simulation using NYURay in the NYU MakerSpace factory using a material-labeled 3D environment model, where each surface carries a material label (e.g., concrete, glass, metal, wood) mapped to frequency-dependent EM properties via ITU-R P.2040 ITU-P2040-3. Traced ray paths from transmitter (red star) to receiver (green circle) demonstrate reflection, penetration, and diffraction propagation mechanisms.
• Figure 2: HoRAMA workflow for automated 3D reconstruction and material classification from RGB video. The pipeline processes video through dense point cloud generation (MASt3R-SLAM), semantic segmentation (PTv3), VLM material classification (Qwen3-VL), and surface reconstruction.
• Figure 3: HoRAMA automated 3D reconstruction from raw point cloud to RT-ready model: (a) Raw dense point cloud from MASt3R-SLAM with color-coded surface normals with noise and outliers. (b) Cleaned point cloud with color-coded surface normals after outlier filtering. (c) Partial view of PTv3 segmented indoor factory environment before smoothing and surface reconstruction. (d) Final 3D environment map with smooth planar surfaces and VLM-classified material labels from ITU-R P.2040 categories (concrete, wood, metal, glass, plywood).
• Figure 4: NYU MakerSpace Factory floor plan showing 12 TX-RX location pairs for channel measurements at 6.75 GHz and 16.95 GHz. Stars represent TX locations, and matching colored circles represent corresponding RX locations for each TX.