Factorization-in-Loop: Proximal Fill-in Minimization for Sparse Matrix Reordering

TL;DR

This work addresses the NP-hard problem of minimizing fill-ins during sparse LU factorization by learning a reordering that minimizes the $l_1$ norm of the Cholesky factor $L$ under the constraint $A_{\theta}=LL^{T}$. It introduces Proximal Fill-in Minimization (PFM), a framework that couples a graph neural network-based reordering with a differentiable matrix reordering layer and a factorization-enhanced loss optimized via ADMM and proximal gradient steps, enabling end-to-end training. Two differentiable permutation schemes—the Gaussian rank distribution and Gumbel-Sinkhorn—produce $P_{\theta}$ for reordering, bridging the gap between node scores and the actual permutation. Experiments on the SuiteSparse collection show substantial reductions in fill-ins and LU time compared with state-of-the-art baselines, with strong scalability and cross-domain generalization across scientific computing problems.

Abstract

Fill-ins are new nonzero elements in the summation of the upper and lower triangular factors generated during LU factorization. For large sparse matrices, they will increase the memory usage and computational time, and be reduced through proper row or column arrangement, namely matrix reordering. Finding a row or column permutation with the minimal fill-ins is NP-hard, and surrogate objectives are designed to derive fill-in reduction permutations or learn a reordering function. However, there is no theoretical guarantee between the golden criterion and these surrogate objectives. Here we propose to learn a reordering network by minimizing \(l_1\) norm of triangular factors of the reordered matrix to approximate the exact number of fill-ins. The reordering network utilizes a graph encoder to predict row or column node scores. For inference, it is easy and fast to derive the permutation from sorting algorithms for matrices. For gradient based optimization, there is a large gap between the predicted node scores and resultant triangular factors in the optimization objective. To bridge the gap, we first design two reparameterization techniques to obtain the permutation matrix from node scores. The matrix is reordered by multiplying the permutation matrix. Then we introduce the factorization process into the objective function to arrive at target triangular factors. The overall objective function is optimized with the alternating direction method of multipliers and proximal gradient descent. Experimental results on benchmark sparse matrix collection SuiteSparse show the fill-in number and LU factorization time reduction of our proposed method is 20% and 17.8% compared with state-of-the-art baselines.

Figures (5)

• Figure 1: Illustration of introducing Cholesky factorization process into the network optimization process.
• Figure 2: Proximal Fill-in Minimization Framework
• Figure 3: Score and Rank Distribution Reparameterization.
• Figure 4: Performance Variation along with Matrix Size for Matrix reordering Methods on SuiteSparse.
• Figure 5: Architecture of the Graph Node Encoder