Factor Fitting, Rank Allocation, and Partitioning in Multilevel Low Rank Matrices

TL;DR

The paper introduces multilevel low rank (MLR) matrices, defined as a row/column-permuted sum of level-wise block-diagonal, low-rank blocks, and demonstrates that MLR retains storage and mat-vec efficiency akin to standard low-rank models. It develops heuristic yet effective factor-fitting and rank-allocation strategies, notably alternating least squares and block-coordinate descent, plus a rank-exchange mechanism to reallocate a fixed total rank across levels. A symmetric and PSD specialization, along with a generalization to GMLR, broadens the framework, while spectral bi-clustering guides hierarchical partitions. Numerical experiments across covariance, Fiedler, DGT, distance, and kernel matrices show MLR achieving superior or competitive fit with favorable storage, often beating HODLR and Monarch baselines. An accompanying open-source package provides practical tools for applying these methods to large, structured matrices in science and engineering.

Abstract

We consider multilevel low rank (MLR) matrices, defined as a row and column permutation of a sum of matrices, each one a block diagonal refinement of the previous one, with all blocks low rank given in factored form. MLR matrices extend low rank matrices but share many of their properties, such as the total storage required and complexity of matrix-vector multiplication. We address three problems that arise in fitting a given matrix by an MLR matrix in the Frobenius norm. The first problem is factor fitting, where we adjust the factors of the MLR matrix. The second is rank allocation, where we choose the ranks of the blocks in each level, subject to the total rank having a given value, which preserves the total storage needed for the MLR matrix. The final problem is to choose the hierarchical partition of rows and columns, along with the ranks and factors. This paper is accompanied by an open source package that implements the proposed methods.

Figures (9)

• Figure 1: Comparison of BCD and ALS. For ALS, one iteration is approximately minimizing over $B$ and then over $C$. For BCD, one iteration is one V-epoch.
• Figure 2: Fitting error during rank allocation, starting from three different initial allocations of rank.
• Figure 3: GICS hierarchical partition of 5000 companies.
• Figure 4: Rank $r=30$ partitioning across $L=6$ levels during fitting for asset covariance matrix with GICS hierarchy, starting from the bottom level $l=6$, uniform and the top level $l=1$ initial allocation.
• Figure 5: Rank $r=30$ partitioning across $L=14$ levels during fitting for asset covariance matrix with full hierarchy, starting from the bottom level $l=14$, uniform ($r_{1}=4$, $r_2 = \cdots =r_{14}=2$) and the top level $l=1$ initial allocation.