Learning to Solve Multiresolution Matrix Factorization by Manifold Optimization and Evolutionary Metaheuristics

TL;DR

This work tackles graph learning in settings with multiscale structure by replacing global Laplacian eigenspaces with a learnable Multiresolution Matrix Factorization (MMF). It combines gradient-based Stiefel-manifold optimization for simultaneous rotation updates with evolutionary metaheuristics to select nested index sets, producing a high-quality MMF wavelet basis. Building on this basis, the authors define Wavelet Neural Networks (WNNs) for graphs, achieving state-of-the-art or competitive results on molecular graph classification and node classification on citation graphs, while demonstrating strong matrix-factorization performance against greedy MMF and Nyström baselines. The approach provides a scalable, sparse, and local spectral framework that can integrate with broader learning pipelines and is released with an open-source implementation for reproducibility and adoption.

Abstract

Multiresolution Matrix Factorization (MMF) is unusual amongst fast matrix factorization algorithms in that it does not make a low rank assumption. This makes MMF especially well suited to modeling certain types of graphs with complex multiscale or hierarchical strucutre. While MMF promises to yields a useful wavelet basis, finding the factorization itself is hard, and existing greedy methods tend to be brittle. In this paper, we propose a ``learnable'' version of MMF that carfully optimizes the factorization using metaheuristics, specifically evolutionary algorithms and directed evolution, along with Stiefel manifold optimization through backpropagating errors. We show that the resulting wavelet basis far outperforms prior MMF algorithms and gives comparable performance on standard learning tasks on graphs. Furthermore, we construct the wavelet neural networks (WNNs) learning graphs on the spectral domain with the wavelet basis produced by our MMF learning algorithm. Our wavelet networks are competitive against other state-of-the-art methods in molecular graphs classification and node classification on citation graphs. We release our implementation at https://github.com/HySonLab/LearnMMF

Figures (9)

• Figure 1: Visualization of the nested set selection process for a $4 \times 4$ matrix $\mathbf{A}$ with $L = 3$ and $k = 2$. The process, depicted from left to right, demonstrates the trimming of the set ${\mathbb{S}}$. The sets ${\mathbb{T}}_\ell$ and ${\mathbb{I}}_\ell$ are chosen by metaheuristics, while the orthogonal transform ${\mathbb{U}}_\ell$ rotates all $k$ coordinates in ${\mathbb{T}}_\ell \cup {\mathbb{I}}_\ell$.
• Figure 2: One-point crossover with common and unique values.
• Figure 3: Metaheuristics convergence for Karate Club data. Selection process based on Evolutionary Algorithm (EA) and Directed Evolution (DE) outperforms the original heuristics proposed by pmlr-v32-kondor14.
• Figure 4: Visualization of some of the wavelets on the Cayley tree of 46 vertices. The low index wavelets (low $\ell$) are highly localized, whereas the high index ones are smoother and spread out over large parts of the graph.
• Figure 5: Matrix factorization for the Karate network (left), Kronecker matrix (middle), and Cayley tree (right). Our learnable MMF consistently outperforms the classic greed methods.

Theorems & Definitions (10)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Lemma 1
• proof
• Lemma 2
• proof
• Lemma 3
• proof