Textured Mesh Saliency: Bridging Geometry and Texture for Human Perception in 3D Graphics

TL;DR

This paper addresses the gap in 3D saliency research by focusing on textured meshes, which combine detailed texture patterns with geometry. It introduces a VR-based eye-tracking dataset of 100 textured meshes, collected from 30 participants, and a texture-aware saliency model consisting of a texture alignment module, a geometric extraction module, and an aggregation module to predict per-face saliency. The results show that integrating texture with geometry improves saliency prediction, while vertex colors alone can degrade performance, underscoring the synergistic role of texture patterns and mesh structure. The work enables more accurate perceptual optimization for high-detail rendering and real-time VR applications, with potential benefits for content creation and mesh simplification guided by human attention. A textured-mesh saliency dataset and the proposed modeling framework offer a new direction for data-driven 3D content processing and perceptual optimization.

Abstract

Textured meshes significantly enhance the realism and detail of objects by mapping intricate texture details onto the geometric structure of 3D models. This advancement is valuable across various applications, including entertainment, education, and industry. While traditional mesh saliency studies focus on non-textured meshes, our work explores the complexities introduced by detailed texture patterns. We present a new dataset for textured mesh saliency, created through an innovative eye-tracking experiment in a six degrees of freedom (6-DOF) VR environment. This dataset addresses the limitations of previous studies by providing comprehensive eye-tracking data from multiple viewpoints, thereby advancing our understanding of human visual behavior and supporting more accurate and effective 3D content creation. Our proposed model predicts saliency maps for textured mesh surfaces by treating each triangular face as an individual unit and assigning a saliency density value to reflect the importance of each local surface region. The model incorporates a texture alignment module and a geometric extraction module, combined with an aggregation module to integrate texture and geometry for precise saliency prediction. We believe this approach will enhance the visual fidelity of geometric processing algorithms while ensuring efficient use of computational resources, which is crucial for real-time rendering and high-detail applications such as VR and gaming.

Figures (8)

• Figure 1: In textured meshes, a texture is a 2D image that, through UV mapping, is accurately projected onto the faces of the mesh, adding visual details and realism. UV coordinates define each vertex's position on the texture image, ensuring that the texture is correctly mapped onto the 3D surface of the mesh.
• Figure 2: VR eye-tracking experiment for textured meshes. (a) illustrates the experimental setup, where the textured meshes are placed at the center of a spherical space and rotated clockwise. (b) shows the sequence of content displayed during the data collection process. (c) presents the recording process of fixation points during the mesh rotation.
• Figure 3: The intersection between the gaze ray and the textured mesh model. The yellow dots represent the gaze fixation triangles where the gaze lingers on the model. The saliency map is presented in the form of a heatmap.
• Figure 4: Model architecture. The network for texture and UV map inputs is the texture alignment module, while the network that processes the geometry inputs is the geometric extraction module.
• Figure 5: UV mapping grid and implicit interpolation for texture feature alignment.