Extending QAOA-GPT to Higher-Order Quantum Optimization Problems

TL;DR

This work extends the QAOA-GPT framework to Higher-Order Unconstrained Binary Optimization (HUBO) problems with cubic terms in the cost Hamiltonian $H_C$. Using FEATHER graph embeddings and training on ADAPT-QAOA-generated circuits for 8- and 16-qubit heavy-hex topologies, the decoder-only Transformer autonomously generates adaptive QAOA–like circuits and variational parameters in a single forward pass. The extended model achieves average approximation ratios above $0.95$ for 16-qubit HUBO instances, closely matching classical ADAPT-QAOA benchmarks while offering orders-of-magnitude faster inference and consistent parameter distributions across depths. The results indicate that generative modeling can scale to complex energy landscapes and serve as a scalable pathway for autonomous quantum circuit design in the NISQ era, with potential extensions to broader Hamiltonians and hardware validation.

Abstract

The recently proposed QAOA-GPT framework demonstrated that generative pre-trained transformers can learn mappings between problem graphs and optimized quantum circuits for the Quantum Approximate Optimization Algorithm (QAOA). In this work, we extend QAOA-GPT to Higher-Order Unconstrained Binary Optimization (HUBO) problems, focusing on spin-glass Hamiltonians that include cubic interaction terms. Using FEATHER graph embeddings to encode topological information, we train the model on graph-circuit pairs generated via ADAPT-QAOA and evaluate its performance on 8- and 16-qubit instances embedded on heavy-hex lattices. The generative model produces adaptive QAOA-like circuits and corresponding variational parameters in a single forward pass, bypassing the iterative classical optimization loop. The generated circuits achieve average approximation ratios exceeding 0.95, closely matching classically optimized ADAPT-QAOA results, while maintaining consistent parameter distributions across circuit depths. These results demonstrate that QAOA-GPT generalizes effectively to higher-order cost Hamiltonians and complex energy landscapes, establishing generative modeling as a scalable pathway toward autonomous variational circuit design and quantum algorithm discovery in the NISQ era.

Figures (9)

• Figure 1: Heavy-hex graph topologies used for spin glass problem generation. (a) 8-qubit graph. (b) 16-qubit graph. Nodes represent qubits (linear terms $Z_v$ on nodes in $V$), edges represent quadratic interactions ($Z_iZ_j$) along edges in $E$, and hyperedges in $W$ encircling three neighboring nodes represent cubic interaction terms ($Z_iZ_jZ_k$) of the HUBO Hamiltonian. All Ising coefficients $d_v, d_{ij}, d_{ijk}$ were uniformly sampled from $\{-1,+1\}$.
• Figure 2: Performance distribution of QAOA-GPT on 200 16-qubit spin glass instances at depth $p_{\text{max}} = 15$. For each instance, the model generated 10 circuits. The 'Best QAOA-GPT Approx. Ratio' (blue) shows the highest approximation ratio ($\alpha$) achieved per instance, averaging $0.9614 \pm 0.0281$. The 'Average QAOA-GPT Approx. Ratio' (orange) shows the mean performance across all 10 generated circuits per instance, averaging $0.9496 \pm 0.0305$. The dashed line indicates the target approximation ratio of 0.97.
• Figure 3: Performance distribution of QAOA-GPT on 200 16-qubit spin glass instances at constrained depth $p_{\text{max}} = 5$. For each instance, the model generated 10 circuits. The 'Best QAOA-GPT Approx. Ratio' (blue) shows the highest approximation ratio ($\alpha$) achieved per instance, averaging $0.8623 \pm 0.0520$. The 'Average QAOA-GPT Approx. Ratio' (orange) shows the mean performance across all 10 generated circuits per instance, averaging $0.8552 \pm 0.0530$. The dashed line indicates the target approximation ratio of 0.97.
• Figure 4: Performance distribution of QAOA-GPT on 200 8-qubit spin glass instances at depth $p_{\text{max}} = 15$. For each instance, the model generated 10 circuits. The 'Best QAOA-GPT Approx. Ratio' (blue) shows the highest approximation ratio ($\alpha$) achieved per instance, averaging $0.9391 \pm 0.0363$. The 'Average QAOA-GPT Approx. Ratio' (orange) shows the mean performance across all 10 generated circuits per instance, averaging $0.8403 \pm 0.0508$. The dashed line indicates the target approximation ratio of 0.97.
• Figure 5: Performance distribution of QAOA-GPT on 200 8-qubit spin glass instances at constrained depth $p_{\text{max}} = 5$. For each instance, the model generated 10 circuits. The 'Best QAOA-GPT Approx. Ratio' (blue) shows the highest approximation ratio ($\alpha$) achieved per instance, averaging $0.8427 \pm 0.0620$. The 'Average QAOA-GPT Approx. Ratio' (orange) shows the mean performance across all 10 generated circuits per instance, averaging $0.8147 \pm 0.0636$. The dashed line indicates the target approximation ratio of 0.97.