Constrained Portfolio Optimization via Quantum Approximate Optimization Algorithm (QAOA) with XY-Mixers and Trotterized Initialization: A Hybrid Approach for Direct Indexing

TL;DR

This paper tackles the challenge of cardinality-constrained portfolio optimization within Direct Indexing, where selecting exactly $K$ assets from $N$ is computationally hard under ESG constraints. It introduces a constraint-preserving QAOA framework that uses Dicke-state initialization and an XY-mixer to stay within the feasible $K$-of-$N$ subspace, augmented by a trotterized parameter initialization to mitigate barren plateaus. The approach is evaluated in an end-to-end pipeline against Simulated Annealing and Hierarchical Risk Parity, using a 2025 walk-forward on 10 US equities with a monthly rebalancing cadence, and shows a Sharpe ratio of $1.81$ and a total return of $30.09\%$, outperforming baselines despite higher turnover. The results suggest that constraint-preserving QAOA can provide superior risk-adjusted performance in liquid, low-cost markets, though turnover costs necessitate turnover-aware objective extensions for practical deployment. Key contributions include (i) a hard-constraint QAOA formulation with Dicke initialization and XY-mixers, (ii) a trotterized warm-start strategy to maintain trainability up to depth $p=6$, and (iii) an end-to-end backtest framework that benchmarks quantum-inspired solvers against industry baselines under realistic transaction costs. This work lays groundwork for integrating quantum-inspired solvers into Direct Indexing platforms, with future work focused on scalability, hardware-noise robustness, and explicit multi-period turnover penalties.

Abstract

Portfolio optimization under strict cardinality constraints is a combinatorial challenge that defies classical convex optimization techniques, particularly in the context of "Direct Indexing" and ESG-constrained mandates. In the Noisy Intermediate-Scale Quantum (NISQ) era, the Quantum Approximate Optimization Algorithm (QAOA) offers a promising hybrid approach. However, standard QAOA implementations utilizing transverse field mixers often fail to strictly enforce hard constraints, necessitating soft penalties that distort the energy landscape. This paper presents a comprehensive analysis of a constraint-preserving QAOA formulation against Simulated Annealing (SA) and Hierarchical Risk Parity (HRP). We implement a specific QAOA ansatz utilizing a Dicke state initialization and an XY-mixer Hamiltonian that strictly preserves the Hamming weight of the solution, ensuring only valid portfolios of size K are explored. Furthermore, we introduce a Trotterized parameter initialization schedule inspired by adiabatic quantum computing to mitigate the "Barren Plateau" problem. Backtesting on a basket of 10 US equities over 2025 reveals that our QAOA approach achieves a Sharpe Ratio of 1.81, significantly outperforming Simulated Annealing (1.31) and HRP (0.98). We further analyze the operational implications of the algorithm's high turnover (76.8%), discussing the trade-offs between theoretical optimality and implementation costs in institutional settings.

Figures (6)

• Figure 1: End-to-end experimental pipeline. Stage 1 estimates $\mu$ and $\Sigma$ on a rolling $L=180$ day window (Ledoit–Wolf covariance) and sets the cardinality constraint $K=5$. Stage 2 performs constraint-preserving QAOA-XY selection using Dicke-state initialization and an XY mixer, with a trotterized parameter warm-start and classical optimization. Candidate portfolios are obtained by measurement, filtered to $|x|=K$ and $P(x)\ge 1\%$, and rescored classically using Eq. \ref{['1']}. Stage 3 allocates continuous weights via SLSQP (Sharpe-max) under box constraints and evaluates performance net of turnover-based transaction costs.
• Figure 2: (a) Cost minimization, (b) gradient magnitudes, and (c) computational cost, as a function of QAOA's circuit depth.
• Figure 3: (a) Portfolio value evolution and (b) Drawdown analysis per month of 2025 for each model.
• Figure 4: The risk-return profile for 2025, for the three models. QAOA exhibits the highest total return from the three models, and for lower volatility than SA. As expected, the HRP results in the lowest volatility but with significantly lower return, as well.
• Figure 5: (a) Average monthly turnover by strategy, (b) Turnover over time for 2025 per strategy.