HybridMQA: Exploring Geometry-Texture Interactions for Colored Mesh Quality Assessment

TL;DR

HybridMQA tackles colored mesh quality assessment by unifying model-based 3D geometry with projection-based texture representations. It introduces a base encoder and graph neural network to learn 3D surface features, a differentiable feature rendering pipeline to align 3D features with colored projections, and a cross-attentive quality encoder to model geometry-texture interactions. The method achieves state-of-the-art performance across four public color MQA datasets, with consistent SRCC/PLCC gains and strong generalization, while providing interpretable visualizations via GradCAM and gMAD analyses. This hybrid, interaction-aware framework offers a practical path toward more perceptually accurate mesh quality assessment and can inform perceptually guided mesh compression and restoration workflows.

Abstract

Mesh quality assessment (MQA) models play a critical role in the design, optimization, and evaluation of mesh operation systems in a wide variety of applications. Current MQA models, whether model-based methods using topology-aware features or projection-based approaches working on rendered 2D projections, often fail to capture the intricate interactions between texture and 3D geometry. We introduce HybridMQA, a first-of-its-kind hybrid full-reference colored MQA framework that integrates model-based and projection-based approaches, capturing complex interactions between textural information and 3D structures for enriched quality representations. Our method employs graph learning to extract detailed 3D representations, which are then projected to 2D using a novel feature rendering process that precisely aligns them with colored projections. This enables the exploration of geometry-texture interactions via cross-attention, producing comprehensive mesh quality representations. Extensive experiments demonstrate HybridMQA's superior performance across diverse datasets, highlighting its ability to effectively leverage geometry-texture interactions for a thorough understanding of mesh quality. Our implementation will be made publicly available.

Figures (11)

• Figure 1: Interactions between texture and geometry. top: complex texture makes the geometry distortion imperceptible. bottom: modifying the geometry affects the appearance of texture distortion. Right half (gray) of each object represents geometry.
• Figure 2: Reference and distorted meshes under geometry distortions. Although the distorted meshes (hawk and bowl) have distinct visual qualities (different mean opinion scores (MOS)), Graphics-LPIPS glpips assigns similar scores and reverses their ranking. HybridMQA aligns well with human perception as it understands the mesh's geometry properties. Blue boxes denote the same regions across viewpoints.
• Figure 3: (a) Overview of HybridMQA. In the model branch, a base encoder extracts 3D features from the mesh's 2D maps, initializing a mesh graph. A GCN extracts 3D surface representations, which are rendered as 2D projections aligned with the colored projections from the texture branch. A quality encoder then captures geometry-texture interactions between the two branches, producing the final mesh quality representation. (b) Full-reference (FR) quality regression, where the absolute difference of HybridMQA's mesh quality representations for reference and distorted meshes is mapped to a quality score via a fully connected network (FCN).
• Figure 4: (a) The quality encoder. The 3D feature and color projections are divided into valid aligned patches and fed into respective encoders to obtain multiscale 3D surface and color representations. The cross-attention modules capture interactions between these representations, which are then concatenated with 3D feature embeddings directly extracted from the patches to form the final mesh quality representation. (b) The cross-attention module consists of two transformer blocks, where we switch the roles of the two inputs.
• Figure 5: SRCC/PLCC performance of HybridMQA and Graphics-LPIPS on various distortion types of SJTU-TMQA dataset. Green distortions affect the geometry or both geometry and texture, while others only impact the texture.