SM3D: Mitigating Spectral Bias and Semantic Dilution in Point Cloud State Space Models

TL;DR

SM3D tackles the intrinsic spectral low-pass bias of State Space Models when applied to serialized point clouds by introducing a Geometric Spectral Compensator (GSC) that injects graph-guided high-frequency geometry and a Semantic Coherence Refiner (SCR) that enforces global semantic stability via spectral anchoring. SCR has two instantiations, SCR-L (exact eigendecomposition) and SCR-C (Chebyshev polynomial approximation), enabling a precision–efficiency trade-off across tasks and scales. Empirical results across ModelNet40, ShapeNetPart, ScanObjectNN, and S3DIS demonstrate state-of-the-art accuracy and robust generalization, with compelling improvements under perturbations and few-shot settings. The approach provides a unified spectral perspective for enhancing Mamba-based point-cloud understanding, balancing geometric fidelity with semantic coherence for practical 3D perception systems.

Abstract

Point clouds are a fundamental 3D data representation that underpins various computer vision tasks. Recently, Mamba has demonstrated strong potential for 3D point cloud understanding. However, existing approaches primarily focus on point serialization, overlooking a more fundamental limitation: State Space Models (SSMs) inherently exhibit a spectral low-pass bias arising from their recursive formulation. In serialized point clouds, this bias is particularly detrimental, as it suppresses high-frequency geometric structures and progressively dilutes semantic discriminability across deep layers. To address these limitations, we propose SM3D, a spectral-aware framework designed to jointly preserve geometric fidelity and semantic consistency. First, a Geometric Spectral Compensator (GSC) is introduced to counteract the low-pass bias by explicitly injecting graph-guided high-frequency components through local Laplacian analysis, thereby restoring structural sensitivity. Second, we design a Semantic Coherence Refiner (SCR) to rectify semantic drift through frequency-aware channel recalibration. To balance theoretical precision and computational efficiency, SCR is instantiated via two pathways: an exact Laplacian eigendecomposition (SCR-L) and a linear-complexity Chebyshev polynomial approximation (SCR-C). Extensive experiments demonstrate that SM3D achieves state-of-the-art performance, including 96.0% accuracy on ModelNet40 and 86.5% mIoU on ShapeNetPart, validating its effectiveness in mitigating spectral low-pass bias and semantic dilution (Code: https://github.com/L1277471578/SM3D).

Figures (8)

• Figure 1: Illustration of SM3D mitigating spectral low-pass bias and semantic dilution. The upper branch illustrates feature blurring caused by the spectral bias of SSMs and semantic dilution. The lower branch highlights the high frequency spectral compensation and semantic coherence.
• Figure 2: Overview of SM3D. (a) Overall architecture showing the embedding,encoder,and decoder structures. (b) Encoder Block core consists of GSC, SCR and two SSM branches: standard forward SSM PointMamba (F‑SSM) and channel‑flipped backward SSM Mamba3D (C-SSM). (c) GSC, extracts high frequency geometric residual. (d) SCR, constructs global spectral anchors. (e) The spectral path in SCR‑C, approximates spectral filtering via Chebyshev polynomials. Symbols: © denotes channel concatenation, $\oplus$ residual addition, $\otimes$ matrix multiplication, and $\odot$ element-wise multiplication.
• Figure 3: Qualitative results of part segmentation by SM3D-L on ShapeNetPart. The top row is the Ground Truth (GT).
• Figure 4: Computational efficiency analysis of SM3D. Left: Inference time on NVIDIA TiTan RTX. Right: GPU memory usage for per sample.
• Figure 5: Qualitative visualizations on ModelNet40. Row 1: Grad-CAM activation maps. Row 2: High frequency spectral energy maps. Row 3: GSC geometric weights.