Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

No-rank Tensor Decomposition Using Metric Learning

Maryam Bagherian

TL;DR

The paper tackles the limitations of traditional tensor decomposition, which relies on rank specifications and reconstruction fidelity, by introducing a no-rank tensor decomposition framework grounded in metric learning. It replaces reconstruction with a discriminative objective built from triplet loss plus regularizers to induce semantically meaningful embeddings that reflect class and structural relationships. The authors provide convergence and geometric guarantees, and demonstrate superior clustering and generalization across face recognition, brain connectivity, and simulated scientific datasets, particularly in data-scarce settings. The approach emphasizes semantic relevance over pixel-level reconstruction and offers computational advantages, presenting a robust alternative to transformer-based models in domains with limited labeled data.

Abstract

Tensor decomposition faces fundamental challenges in analyzing high-dimensional data, where traditional methods based on reconstruction and fixed-rank constraints often fail to capture semantically meaningful structures. This paper introduces a no-rank tensor decomposition framework grounded in metric learning, which replaces reconstruction objectives with a discriminative, similarity-based optimization. The proposed approach learns data-driven embeddings by optimizing a triplet loss with diversity and uniformity regularization, creating a feature space where distance directly reflects semantic similarity. We provide theoretical guarantees for the framework's convergence and establish bounds on its metric properties. Evaluations across diverse domains -- including face recognition (LFW, Olivetti), brain connectivity analysis (ABIDE), and simulated data (galaxy morphology, crystal structures) -- demonstrate that our method outperforms baseline techniques, including PCA, t-SNE, UMAP, and tensor decomposition baselines (CP and Tucker). Results show substantial improvements in clustering metrics (Silhouette Score, Davies-Bouldin Index, Calinski-Harabasz Index, Separation Ratio, Adjusted Rand Index, Normalized Mutual Information) and reveal a fundamental trade-off: while metric learning optimizes global class separation, it deliberately transforms local geometry to align with semantic relationships. Crucially, our approach achieves superior performance with smaller training datasets compared to transformer-based methods, offering an efficient alternative for domains with limited labeled data. This work establishes metric learning as a paradigm for tensor-based analysis, prioritizing semantic relevance over pixel-level fidelity while providing computational advantages in data-scarce scenarios.

No-rank Tensor Decomposition Using Metric Learning

TL;DR

The paper tackles the limitations of traditional tensor decomposition, which relies on rank specifications and reconstruction fidelity, by introducing a no-rank tensor decomposition framework grounded in metric learning. It replaces reconstruction with a discriminative objective built from triplet loss plus regularizers to induce semantically meaningful embeddings that reflect class and structural relationships. The authors provide convergence and geometric guarantees, and demonstrate superior clustering and generalization across face recognition, brain connectivity, and simulated scientific datasets, particularly in data-scarce settings. The approach emphasizes semantic relevance over pixel-level reconstruction and offers computational advantages, presenting a robust alternative to transformer-based models in domains with limited labeled data.

Abstract

Tensor decomposition faces fundamental challenges in analyzing high-dimensional data, where traditional methods based on reconstruction and fixed-rank constraints often fail to capture semantically meaningful structures. This paper introduces a no-rank tensor decomposition framework grounded in metric learning, which replaces reconstruction objectives with a discriminative, similarity-based optimization. The proposed approach learns data-driven embeddings by optimizing a triplet loss with diversity and uniformity regularization, creating a feature space where distance directly reflects semantic similarity. We provide theoretical guarantees for the framework's convergence and establish bounds on its metric properties. Evaluations across diverse domains -- including face recognition (LFW, Olivetti), brain connectivity analysis (ABIDE), and simulated data (galaxy morphology, crystal structures) -- demonstrate that our method outperforms baseline techniques, including PCA, t-SNE, UMAP, and tensor decomposition baselines (CP and Tucker). Results show substantial improvements in clustering metrics (Silhouette Score, Davies-Bouldin Index, Calinski-Harabasz Index, Separation Ratio, Adjusted Rand Index, Normalized Mutual Information) and reveal a fundamental trade-off: while metric learning optimizes global class separation, it deliberately transforms local geometry to align with semantic relationships. Crucially, our approach achieves superior performance with smaller training datasets compared to transformer-based methods, offering an efficient alternative for domains with limited labeled data. This work establishes metric learning as a paradigm for tensor-based analysis, prioritizing semantic relevance over pixel-level fidelity while providing computational advantages in data-scarce scenarios.

Paper Structure

This paper contains 27 sections, 1 theorem, 40 equations, 8 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Metric Learning Framework
  4. Theoretical Framework
  5. Triplet Loss with Regularization
  6. Locality Preservation Framework
  7. Network Architecture
  8. Convergence and Geometric Guarantees
  9. Metrics and Methodology Analysis
  10. Evaluation Metrics
  11. Visualization of High-Dimensional Embeddings
  12. Triplet Mining Strategy Analysis
  13. Results
  14. Metric Learning Performance in Face Recognition
  15. Metric Learning Performance on Brain Connectivity Data
  16. ...and 12 more sections

Key Result

Theorem 3

Under the assumptions of Lemmas lemma:convergence and lemma:structure, the proposed metric learning framework: This result bridges optimization guarantees with geometric structure, ensuring that the learned representation is both stable and semantically meaningful.

Figures (8)

  • Figure 1: t-SNE visualization of face embeddings. Left to right: Olivetti (original data), Olivetti (metric learning), LFW (original data), LFW (metric learning). Metric learning produces distinct, identity-based clusters.
  • Figure 2: Nearest-neighbor retrieval on the LFW dataset using the metric learning embedding. The model correctly identifies same-identity images as the closest neighbors, with small corresponding Euclidean distances.
  • Figure 3: Nearest-neighbor retrieval on the Olivetti Faces dataset using the metric learning embedding. The results demonstrate robust, identity-based retrieval.
  • Figure 4: Example images from simulated datasets. Top: Galaxy morphology classes (Elliptical, Spiral, Lenticular, Irregular). Bottom: Crystal structure classes (Cubic, Hexagonal, Tetragonal, Orthorhombic).
  • Figure 5: Embedding visualizations. Two Top Rows Galaxy morphology dataset. Two Bottom Rows Crystal structure dataset. Metric learning produces the most distinct and compact clusters, demonstrating its effectiveness in learning semantically meaningful representations.
  • ...and 3 more figures

Theorems & Definitions (5)

  • proof
  • proof
  • Theorem 3
  • proof
  • Remark 4