Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonnegative Tensor Decomposition Via Collaborative Neurodynamic Optimization

Salman Ahmadi-Asl, Valentin Leplat, Anh-Huy Phan, Andrzej Cichocki

TL;DR

This paper addresses the challenge of computing nonnegative CPD by introducing a collaborative neurodynamic optimization framework in which multiple recurrent neural networks communicate through particle swarm optimization to jointly search for a (nearly) global minimum. The approach combines continuous-time and discrete-time projection neural networks with Hessian-based preconditioning and optional log-barrier regularization to enforce nonnegativity, along with PSO and diversity mechanisms to escape local minima. Key contributions include the formulation of CNO-CPD and CNO-DTPNN, convergence guarantees with probability one under PSO for the global minimum, and extensive empirical validation on synthetic and real-world tensors (including hyperspectral data and face datasets) that demonstrate improved accuracy and robustness, especially under high collinearity. The work has practical implications for scalable, distributed tensor factorization in areas such as image processing, clustering, and hyperspectral unmixing, and it lays groundwork for extending collaborative neurodynamic models to other tensor decompositions and divergences.

Abstract

This paper introduces a novel collaborative neurodynamic model for computing nonnegative Canonical Polyadic Decomposition (CPD). The model relies on a system of recurrent neural networks to solve the underlying nonconvex optimization problem associated with nonnegative CPD. Additionally, a discrete-time version of the continuous neural network is developed. To enhance the chances of reaching a potential global minimum, the recurrent neural networks are allowed to communicate and exchange information through particle swarm optimization (PSO). Convergence and stability analyses of both the continuous and discrete neurodynamic models are thoroughly examined. Experimental evaluations are conducted on random and real-world datasets to demonstrate the effectiveness of the proposed approach.

Nonnegative Tensor Decomposition Via Collaborative Neurodynamic Optimization

TL;DR

This paper addresses the challenge of computing nonnegative CPD by introducing a collaborative neurodynamic optimization framework in which multiple recurrent neural networks communicate through particle swarm optimization to jointly search for a (nearly) global minimum. The approach combines continuous-time and discrete-time projection neural networks with Hessian-based preconditioning and optional log-barrier regularization to enforce nonnegativity, along with PSO and diversity mechanisms to escape local minima. Key contributions include the formulation of CNO-CPD and CNO-DTPNN, convergence guarantees with probability one under PSO for the global minimum, and extensive empirical validation on synthetic and real-world tensors (including hyperspectral data and face datasets) that demonstrate improved accuracy and robustness, especially under high collinearity. The work has practical implications for scalable, distributed tensor factorization in areas such as image processing, clustering, and hyperspectral unmixing, and it lays groundwork for extending collaborative neurodynamic models to other tensor decompositions and divergences.

Abstract

This paper introduces a novel collaborative neurodynamic model for computing nonnegative Canonical Polyadic Decomposition (CPD). The model relies on a system of recurrent neural networks to solve the underlying nonconvex optimization problem associated with nonnegative CPD. Additionally, a discrete-time version of the continuous neural network is developed. To enhance the chances of reaching a potential global minimum, the recurrent neural networks are allowed to communicate and exchange information through particle swarm optimization (PSO). Convergence and stability analyses of both the continuous and discrete neurodynamic models are thoroughly examined. Experimental evaluations are conducted on random and real-world datasets to demonstrate the effectiveness of the proposed approach.

Paper Structure

This paper contains 11 sections, 9 theorems, 57 equations, 11 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. CNO-based algorithm for nonnegative CPD (CNO-CPD)
  4. Log barrier formulation
  5. Convergence analysis of the CNO-CPD
  6. CPD with Discrete-time projection neural network (DTPNN)
  7. Stability and convergence analysis of DTPNN
  8. Experimental Results
  9. Conclusion
  10. Acknowledgement
  11. appendix

Key Result

Lemma 4.1

yan2014collective The equilibrium points of a one-layer RNN and KKT points of a global constrained optimization problem have one-to-one correspondence.

Figures (11)

  • Figure 1: Polyadic decomposition of tensor ${\bf X}$ of Rank-$R$cichocki2009nonnegative
  • Figure 1: Flowchart of the proposed CNO-CPD method.
  • Figure 1: Comparing the algorithms for decomposing a nonnegative tensor of size $9\times 9\times 9$ and the tensor rank $R=10$.
  • Figure 2: The procedure of collaborative neurodynamic optimization for nonnegative CPD.
  • Figure 2: (left) The numerical results of the continuous neurodynamic for decomposing a nonnegative tensor of size $9\times 9\times 9$ and the tensor ranks $R=11,12,\ldots,16$. (right) Comparing the continuous neurodynamic and its log barrier formulation for decomposing a nonnegative tensor of size $9\times 9\times 9$ and the tensor rank $R=20$.
  • ...and 6 more figures

Theorems & Definitions (15)

  • Definition 2.1
  • Lemma 4.1
  • Lemma 4.2
  • Theorem 4.3
  • Proof 1
  • Lemma 5.1
  • Proof 2
  • Lemma 5.2
  • Lemma 5.3
  • Proof 3
  • ...and 5 more