Efficient Constrained Tensor Factorization by Alternating Optimization with Primal-Dual Splitting
Shunsuke Ono, Takuma Kasai
TL;DR
This work tackles constrained tensor factorization by addressing the limitations of AO-ADMM, notably matrix inversions and restricted support for structured regularizers. It introduces AO-PDS, an approach that embeds primal-dual splitting within alternating optimization to solve per-mode subproblems without inversions and with support for linear operators. Empirical results on regularized nonnegative CPD show AO-PDS is roughly three times faster and often yields better factorization quality at larger ranks. The method broadens the practical applicability of constrained tensor factorization by enabling efficient handling of a wider class of regularizers such as overlapping group penalties and total variation, with potential impact across signal processing and data analysis.
Abstract
Tensor factorization with hard and/or soft constraints has played an important role in signal processing and data analysis. However, existing algorithms for constrained tensor factorization have two drawbacks: (i) they require matrix-inversion; and (ii) they cannot (or at least is very difficult to) handle structured regularizations. We propose a new tensor factorization algorithm that circumvents these drawbacks. The proposed method is built upon alternating optimization, and each subproblem is solved by a primal-dual splitting algorithm, yielding an efficient and flexible algorithmic framework to constrained tensor factorization. The advantages of the proposed method over a state-of-the-art constrained tensor factorization algorithm, called AO-ADMM, are demonstrated on regularized nonnegative tensor factorization.