A Mixture of Experts Vision Transformer for High-Fidelity Surface Code Decoding

TL;DR

This work addresses the challenge of decoding topological quantum error-correcting codes with scalable, real-time performance. It develops QuantumSMoE, a Vision Transformer-based decoder that embeds the toric code's geometry via PlusConv2D, enforces locality with Adaptive Masking, and scales capacity with a SoftMoE layer aided by a slot orthogonality loss. The approach yields superior logical error rates (LER) compared to classical and ML baselines, while maintaining competitive bit error rates (BER) across multiple code distances under depolarizing noise. By explicitly leveraging lattice structure and a scalable mixture-of-experts design, the method demonstrates significant practical potential for real-time quantum error correction and paves the way for applying similar inductive biases to other topological codes and noise models.

Abstract

Quantum error correction is a key ingredient for large scale quantum computation, protecting logical information from physical noise by encoding it into many physical qubits. Topological stabilizer codes are particularly appealing due to their geometric locality and practical relevance. In these codes, stabilizer measurements yield a syndrome that must be decoded into a recovery operation, making decoding a central bottleneck for scalable real time operation. Existing decoders are commonly classified into two categories. Classical algorithmic decoders provide strong and well established baselines, but may incur substantial computational overhead at large code distances or under stringent latency constraints. Machine learning based decoders offer fast GPU inference and flexible function approximation, yet many approaches do not explicitly exploit the lattice geometry and local structure of topological codes, which can limit performance. In this work, we propose QuantumSMoE, a quantum vision transformer based decoder that incorporates code structure through plus shaped embeddings and adaptive masking to capture local interactions and lattice connectivity, and improves scalability via a mixture of experts layer with a novel auxiliary loss. Experiments on the toric code demonstrate that QuantumSMoE outperforms state-of-the-art machine learning decoders as well as widely used classical baselines.

Figures (7)

• Figure 1: Error syndromes in the Toric code. $Z$-type syndrome qubits detect $X$ errors (yellow), while $X$-type syndrome qubits detect $Z$ errors (green); both types respond to $Y$ errors (purple). A single-qubit error typically flips the measurement outcomes of two adjacent syndrome qubits, complicating the decoding process as the error density increases.
• Figure 2: Overall framework of the proposed model. The qubit lattice of the toric code is first partitioned into patches and embedded using the proposed PlusConv2D convolutional layer. Within each Transformer block, the standard MLP is replaced by a SoftMoE layer, where discrepancy among slot representations is further promoted through the introduced auxiliary loss $\mathcal{L}_{\text{os}}$.
• Figure 3: Proposed adaptive masks at code distances $L=4$ and $L=6$.
• Figure 4: Comparison of Bit Error Rate (BER) and Logical Error Rate (LER) between our method and classical baselines, including MWPM, MWPM-Corr, BP-LSD, as well as the ML-based model QECCT. The results are evaluated over 18 distinct physical error rates ranging from $0.05$ to $0.2$. Our model consistently achieves lower LER than both classical and machine learning–based decoders.
• Figure 5: LER comparison for the proposed QuantumSMoE with and without the MoE layer for distance $L=6,8$. QECCT is included as a baseline to demonstrate the effectiveness of patch embedding in extracting local topological information.