Asymptotic Vanishing of the Success Probability in Shor's Algorithm
João P. da Cruz
TL;DR
This work analyzes Shor's quantum factoring algorithm through a measure-theoretic lens, showing that although the local success probability is bounded below by $1/2$ for each fixed modulus $N$, no global asymptotic measure exists as $N\to\infty$ because the uniform measures $\mu_N$ on $\Omega_N=(\mathbb{Z}/N\mathbb{Z})^\times$ are non-tight. The authors prove an asymptotic vanishing of the joint success weight, $p(\mathrm{good},a')=p(\mathrm{good}|a',N)p(a'|N)$, decaying like $1/\varphi(N)$, and corroborate this with Monte Carlo simulations for semiprimes up to $N\le 10^6$ showing the expected $N^{-2}$ scaling since $\varphi(N)\sim 3N^2/\pi^2$. The result implies that the notion of an "expected polynomial time" for order finding is inherently local and cannot extend to the asymptotic regime, revealing a structural limit to the scalability of quantum factoring beyond small instances. Consequently, large-scale implementations of Shor's algorithm are not merely hardware-limited but fundamentally constrained by the non-existence of a global ergodic measure on the arithmetic domain, explaining the lack of cryptanalytic progress at cryptographically relevant sizes.
Abstract
Shor's factoring algorithm guarantees a success probability of at least one half for any fixed modulus N = pq with distinct primes p and q. We show that this guarantee does not extend to the asymptotic regime. As N -> infinity, the multiplicative groups Omega_N = (Z/NZ)^x form a non-tight family of probability spaces, and the probability weight associated with successful bases, proportional to p(success | a', N) p(a' | N), decays as 1/phi(N). The ensemble of uniform measures {mu_N} therefore admits no weak limit, implying an asymptotic loss of ergodicity. Monte Carlo simulations up to N <= 10^6 confirm this decay and the absence of a stationary success probability. These results demonstrate that the "expected polynomial time" in order finding is only locally defined: no global expectation exists once the arithmetic domain expands. The asymptotic vanishing of success probability explains the empirical absence of large-N implementations of Shor's algorithm and sets a fundamental limit on the scalability of quantum factoring.
