MACE: Mixture-of-Experts Accelerated Coordinate Encoding for Large-Scale Scene Localization and Rendering

TL;DR

MACE tackles the challenge of large-scale scene localization and rendering by introducing a Mixture-of-Experts framework that activates a single specialized sub-network per inference, substantially reducing computation while preserving accuracy. The method combines an Auxiliary-Loss-Free Load Balancing strategy, a multimodal Position Decoding scheme, and a gating network to leverage implicit global context without explicit global descriptors. Localization outputs feed a forward rendering pipeline based on Gaussian Splatting, enabling unsupervised, photorealistic rendering with high fidelity and efficiency. Extensive Cambridge Landmarks experiments show MACE achieves superior localization with compact activation maps and renders high-quality views within minutes, demonstrating practical potential for real-time AR on constrained hardware.

Abstract

Efficient localization and high-quality rendering in large-scale scenes remain a significant challenge due to the computational cost involved. While Scene Coordinate Regression (SCR) methods perform well in small-scale localization, they are limited by the capacity of a single network when extended to large-scale scenes. To address these challenges, we propose the Mixed Expert-based Accelerated Coordinate Encoding method (MACE), which enables efficient localization and high-quality rendering in large-scale scenes. Inspired by the remarkable capabilities of MOE in large model domains, we introduce a gating network to implicitly classify and select sub-networks, ensuring that only a single sub-network is activated during each inference. Furtheremore, we present Auxiliary-Loss-Free Load Balancing(ALF-LB) strategy to enhance the localization accuracy on large-scale scene. Our framework provides a significant reduction in costs while maintaining higher precision, offering an efficient solution for large-scale scene applications. Additional experiments on the Cambridge test set demonstrate that our method achieves high-quality rendering results with merely 10 minutes of training.

Figures (6)

• Figure 1: Map Size vs Position Average Error. Comparison with several counterparts on the Cambridge dataset demonstrates that our method reduces activation weight by 72% vs. Poker (ACE $\times$ 4) while maintaining low localization errors.
• Figure 2: Overview of MACE. (a) Localization Pipeline. Input images are encoded via a pretrained CNN, with local features stored in a training buffer. A router, guided by Auxiliary-Loss-Free Load Balancing, selects expert MLPs ($f^t_{\cdot}\text{SCR}$) jointly optimized by reprojection loss to enforce geometric consistency. The MLP output is refined by a Position Decoder to generate final coordinate predictions. (b) Localization-rendering cascade. Better localization via the MACE localization pipeline directly improves rendering quality. (c) The router selects pretrained expert MLPs for coordinate prediction, which are fused with features and point clouds, then processed by a Gaussian head to generate static views optimized with PSNR loss.
• Figure 3: Qualitative comparison across different localization frameworks. Red boxes in ACE highlight artifacts such as misaligned architectural details, emphasizing the superior visual fidelity achieved by MACE through more accurate localization.
• Figure 4: Ablation of MoE architecture. We compare the heatmaps of gating network activations with and without ALF-LB strategy. The left exhibits balanced expert utilization, whereas the right reveals significant imbalance, highlighting ALF-LB's effectiveness in mitigating utilization bias.
• Figure 5: Ablation Comparison of Geometric Priors in AR Rendering. Renderings with monocular depth exhibit severe artifacts like fragmented structures and blackened regions, while MACE - generated renderings closely match the ground truth, demonstrating the superiority of our SCR - based geometric prior in ensuring high - fidelity 3DGS rendering for AR static view tasks.