GeoScaler: Geometry and Rendering-Aware Downsampling of 3D Mesh Textures

TL;DR

GeoScaler tackles the problem of downsampling high-resolution texture maps for 3D meshes by making the downsampling geometry- and UV-aware and optimizing for rendered view fidelity rather than UV-space fidelity. It introduces GeoCoding and UVWarper modules within a DNN that encodes geometry, fuses it with texture information, and locally warps the UV layout, all guided by a differentiable rendering loss. The approach yields sizable perceptual improvements (PSNR/SSIM) over traditional resampling methods at 4x and 8x downsampling across multiple real-world datasets, while enabling more memory-efficient rendering suitable for devices with limited budgets. While effective, the method depends on high-resolution input textures and incurs additional computation time, suggesting future work in faster per-mesh optimization and extending the framework to other texture-like maps.

Abstract

High-resolution texture maps are necessary for representing real-world objects accurately with 3D meshes. The large sizes of textures can bottleneck the real-time rendering of high-quality virtual 3D scenes on devices having low computational budgets and limited memory. Downsampling the texture maps directly addresses the issue, albeit at the cost of visual fidelity. Traditionally, downsampling of texture maps is performed using methods like bicubic interpolation and the Lanczos algorithm. These methods ignore the geometric layout of the mesh and its UV parametrization and also do not account for the rendering process used to obtain the final visualization that the users will experience. Towards filling these gaps, we introduce GeoScaler, which is a method of downsampling texture maps of 3D meshes while incorporating geometric cues, and by maximizing the visual fidelity of the rendered views of the textured meshes. We show that the textures generated by GeoScaler deliver significantly better quality rendered images compared to those generated by traditional downsampling methods

Figures (11)

• Figure 1: Qualitative comparisons of downsampling. GeoScalar can downsample texture maps of 3D meshes while providing significantly improved quality of rendered images as compared to existing methods such as bicubic resampling. The results shown here were computed on the clothdolls mesh from our 3DSet5 mesh dataset.
• Figure 2: The trade-off between warping and discontinuities when creating UV parametrization. (a) Regions like Greenland and Antarctica which lie near the poles consume disproportionately large areas on Mercator projection. (b) Goode homolosine projection reduces warping artifacts but introduces many discontinuities.
• Figure 3: Overview of the GeoScaler architecture. For a given input textured 3D mesh, an optimal texture downsampling function is learned which incorporates information of the geometry of the mesh and its UV parametrization. The model parameters are iteratively updated via differentiable rendering by computing the loss between views rendered from the original texture of the mesh versus the downsampled texture.
• Figure 4: Overview of the GeoCoding module. Features obtained from the texture encoder are mapped to vertices on the surface of the mesh using the UV parametrization of the mesh. Performing graph convolution operations on these mapped features allows combining geometric information from the mesh with texture information, which reduces artifacts at texture map discontinuities. The output features are then interpolated using Barycentric coordinates and reprojected back to the UV plane.
• Figure 5: The newly proposed 3DSet5 dataset. Meshes in the 3DSet5 dataset are unprocessed and unrefined after reconstructing from multiple images of the objects and contain numerous geometric and textural irregularities.