Efficient Tensor Completion Algorithms for Highly Oscillatory Operators

TL;DR

This work tackles the challenge of reconstructing highly oscillatory operators discretized as $n\times n$ matrices, where traditional low-rank methods fail to capture the structure efficiently. By reformulating the butterfly decomposition as a tensor network and reshaping the matrix into a higher-order tensor, the authors enable loss-minimization-based completion using ALS and ADAM with a principled initial guess generated from cheap low-rank approximations. The proposed approach achieves observed-entry requirements on the order of $O(n\log n)$ and empirical computational cost $O(n\log^3 n)$, outperforming low-rank and QTT baselines in seismic-relevant tests on Green's functions, inverse scattering, and Radon transforms. This butterfly tensor completion framework offers substantial speedups and accuracy gains for oscillatory operators and suggests avenues for integration with neural networks and GPU-accelerated kernels.

Abstract

This paper presents low-complexity tensor completion algorithms and their efficient implementation to reconstruct highly oscillatory operators discretized as $n\times n$ matrices. The underlying tensor decomposition is based on the reshaping of the input matrix and its butterfly decomposition into an order $O (\log n)$ tensor. The reshaping of the input matrix into a tensor allows for representation of the butterfly decomposition as a tensor decomposition with dense tensors. This leads to efficient utilization of the existing software infrastructure for dense and sparse tensor computations. We propose two tensor completion algorithms in the butterfly format, using alternating least squares and gradient-based optimization, as well as a novel strategy that uses low-rank matrix completion to efficiently generate an initial guess for the proposed algorithms. To demonstrate the efficiency and applicability of our proposed algorithms, we perform three numerical experiments using simulated oscillatory operators in seismic applications. In these experiments, we use $O (n \log n)$ observed entries in the input matrix and demonstrate an $O(n\log^3 n)$ computational cost of the proposed algorithms, leading to a speedup of orders of magnitudes per iteration for large matrices compared to the low-rank matrix and quantized tensor-train completion. Moreover, the proposed butterfly completion algorithms, equipped with the novel initial guess generation strategy, achieve reconstruction errors that are smaller by an order of magnitude, enabling accurate recovery of the underlying structure compared to the state-of-the-art completion algorithms.

Figures (10)

• Figure 1: Illustration of a $4$-level butterfly matrix decomposition $\bm{X}=\bm{S}^1\bm{S}^2\bm{S}^3\bm{S}^4\bm{S}^5\bm{S}^6$. \newlabelfig:BF_4L0
• Figure 1: The real part of a Green's function matrix with $n=4096$, $c=4$, $L=10$. (a) The ground truth data $\bm{T}$. (b) The observed data $\bm{T}_\Omega$ with $|\Omega|=66n\log n=3244032$. (c) The low-rank reconstruction with rank $r=400$. (d) The butterfly reconstruction using \ref{['alg:ALS_BF']} with rank $r=11$. \newlabelfig:mat_Green0
• Figure 2: (a) A $2$-level butterfly matrix decomposition $\bm{X}=\bm{S}^1\bm{S}^2\bm{S}^3\bm{S}^4$. For a matrix entry $(i,j)$ with $i=2c+a$ and $j=c+b$, $a,b\leq c$, $x_{ij}$ is computed by the product of the subblocks highlighted in blue. (b) The diagram of reshaping the $2$-level butterfly matrix decomposition $\bm{X}$ as an order $2L+2$ tensor $\pmb{\mathcal{X}}$ with tensor cores $\pmb{\mathcal{S}}^1,\pmb{\mathcal{S}}^2,\cdots,\pmb{\mathcal{S}}^{L+1},\pmb{\mathcal{S}}^{L+2}$. Each gray circle represents one tensor core $\pmb{\mathcal{S}}^l$ of order $L+3$ for $1<l<L+2$ or order $L+2$ otherwise. Each black node denotes an identity node, representing an external edge $i_0$, $i_1$, $i_2$ or $j_0$, $j_1$, $j_2$. The $(i,j)$ entry in (a) now becomes $(i_0,i_1,i_2,j_0,j_1,j_2)=(1,0,a,0,1,b)$. (c) The diagram of the QTT representation of the same tensor in (b): $\pmb{\mathcal{X}}$ with tensor cores $\pmb{\mathcal{S}}^1, \pmb{\mathcal{S}}^2, \pmb{\mathcal{S}}^3$ where each tensor core has order 3 or 4, independent of $L$. \newlabelfig:tensor_diagram0
• Figure 2: The convergence of a Green's function example with $n=16384$, $c=4$, $L=12$ using the low-rank matrix completion with rank $r=200,400$, butterfly completion \ref{['alg:ALS_BF']} and \ref{['alg:ADAM_BF']} with rank $r=11$, and QTT completion \ref{['alg:ALS_QTT']} with rank $r=50$. (a) Training error with $|\Omega|=66n\log n$. (b) Test error with $|\Omega_{\rm test}|=11n\log n$. \newlabelfig:convergence_Green_L120
• Figure 3: The convergence of a Green's function example with $n=65536$, $c=4$, $L=14$ using the low-rank matrix completion with rank $r=200$, and butterfly completion \ref{['alg:ALS_BF']} with rank $r=11$. (a) Training error with $|\Omega|=66n\log n$. (b) Test error with $|\Omega_{\rm test}|=11n\log n$. \newlabelfig:convergence_Green_L140