Indefiniteness makes lattice reduction easier
Antoine Joux
TL;DR
The paper extends lattice reduction to indefinite quadratic forms by introducing a general, non-restrictive definition of indefinite lattices and a new polynomial-time algorithm (IndefLLL) that leverages the lattice–quadratic form correspondence. It replaces standard Gram-Schmidt with a generalized GSO, accounts for hyperbolic planes, and reduces 2×2 blocks via indefinite binary quadratic forms, actively promoting sign alternation to tighten reduction while avoiding isotropic vectors unless they form hyperbolic planes. Theoretical results bound the resulting basis in terms of the non-degenerate Gram determinant and the lattice’s signature, and the algorithm’s complexity is handled via a potential framework. Experimental evidence from Magma suggests substantial gains over prior indefinite reductions, especially for higher-signature lattices, indicating that indefiniteness can make lattice reduction easier and that the performance depends strongly on the signature rather than dimension.
Abstract
Since the invention of the famous LLL algorithm, lattice reduction has been an extremely useful tool in computational number theory. By construction, the LLL algorithm deals with lattices living in a vector space endowed with a positive definite scalar product. However, it seems quite nature to ask about the indefinite case, where the scalar product is replaced by an arbitrary quadratic form, possibily indefinite. This question was considered independently in two lines of work. One by G{á}bor Ivanyos and {Á}gnes Sz{á}nt{ó} and one by Denis Simon. Both lead to an algorithm that generalizes LLL and whose performance is very similar to LLL, i.e. a polynomial-time algorithm that approximates the shortest vector within an approximation factor exponential in the dimension. Denis Simon achieves an approximation factor close to that of LLL under the assumption that no isotropic vectors arise during reduction. G{á}bor Ivanyos and {Á}gnes Sz{á}nt{ó} show that it is possible to avoid isotropic vectors altogether, at the cost of a somewhat worse approximation factor. In this paper, we revisit the reduction of indefinite lattices and conclude that it can lead to much better reduced representations that previously thought. We also conclude that the approximation factor depends on the signature of the indefinite lattice rather than on its dimension.