Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neuc-MDS: Non-Euclidean Multidimensional Scaling Through Bilinear Forms

Chengyuan Deng, Jie Gao, Kevin Lu, Feng Luo, Hongbin Sun, Cheng Xin

TL;DR

Neuc-MDS addresses embedding non-Euclidean, non-metric dissimilarities by replacing the Euclidean inner product with a symmetric bilinear form $f(u,v)=u^TAv$ and optimizing over both positive and negative Gram-eigenvalues to minimize $\|\hat{D}-D\|_F^2$. It provides a three-term STRESS decomposition and an efficient $O(n)$ eigenvalue-selection algorithm (EV-Selection), plus a generalized Neuc-MDS$^+$ that optimizes a rank-$k$ transform for tighter lower bounds; random-matrix analysis further clarifies when Neuc-MDS outperform classical MDS. Extensive experiments on synthetic and real data (including CuMiDa genomics and image datasets) demonstrate substantial reductions in STRESS and distortion and resolve the dimensionality paradox, with Neuc-MDS$^+$ offering favorable behavior regarding negative distances. By recovering classical MDS under EDM conditions and accommodating a broad family of dissimilarities (e.g., cosine, KL, etc.), this framework enables principled, scalable non-Euclidean embeddings for a wide range of applications.

Abstract

We introduce Non-Euclidean-MDS (Neuc-MDS), an extension of classical Multidimensional Scaling (MDS) that accommodates non-Euclidean and non-metric inputs. The main idea is to generalize the standard inner product to symmetric bilinear forms to utilize the negative eigenvalues of dissimilarity Gram matrices. Neuc-MDS efficiently optimizes the choice of (both positive and negative) eigenvalues of the dissimilarity Gram matrix to reduce STRESS, the sum of squared pairwise error. We provide an in-depth error analysis and proofs of the optimality in minimizing lower bounds of STRESS. We demonstrate Neuc-MDS's ability to address limitations of classical MDS raised by prior research, and test it on various synthetic and real-world datasets in comparison with both linear and non-linear dimension reduction methods.

Neuc-MDS: Non-Euclidean Multidimensional Scaling Through Bilinear Forms

TL;DR

Neuc-MDS addresses embedding non-Euclidean, non-metric dissimilarities by replacing the Euclidean inner product with a symmetric bilinear form and optimizing over both positive and negative Gram-eigenvalues to minimize . It provides a three-term STRESS decomposition and an efficient eigenvalue-selection algorithm (EV-Selection), plus a generalized Neuc-MDS that optimizes a rank- transform for tighter lower bounds; random-matrix analysis further clarifies when Neuc-MDS outperform classical MDS. Extensive experiments on synthetic and real data (including CuMiDa genomics and image datasets) demonstrate substantial reductions in STRESS and distortion and resolve the dimensionality paradox, with Neuc-MDS offering favorable behavior regarding negative distances. By recovering classical MDS under EDM conditions and accommodating a broad family of dissimilarities (e.g., cosine, KL, etc.), this framework enables principled, scalable non-Euclidean embeddings for a wide range of applications.

Abstract

We introduce Non-Euclidean-MDS (Neuc-MDS), an extension of classical Multidimensional Scaling (MDS) that accommodates non-Euclidean and non-metric inputs. The main idea is to generalize the standard inner product to symmetric bilinear forms to utilize the negative eigenvalues of dissimilarity Gram matrices. Neuc-MDS efficiently optimizes the choice of (both positive and negative) eigenvalues of the dissimilarity Gram matrix to reduce STRESS, the sum of squared pairwise error. We provide an in-depth error analysis and proofs of the optimality in minimizing lower bounds of STRESS. We demonstrate Neuc-MDS's ability to address limitations of classical MDS raised by prior research, and test it on various synthetic and real-world datasets in comparison with both linear and non-linear dimension reduction methods.

Paper Structure

This paper contains 22 sections, 12 theorems, 51 equations, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dimension Reduction with Bilinear Forms
  4. MDS in the Lens of Bilinear Forms
  5. Non-Euclidean MDS
  6. Theoretical Results for Non-Euclidean MDS
  7. Error Analysis
  8. An Optimal Algorithm for Eigenvalue Selection
  9. Analysis on Random Symmetric Matrices
  10. Beyond Binary Selection of Eigenvalues
  11. Experiments
  12. Synthetic Data.
  13. Discussion and Conclusion
  14. Hollow Matrices and Relationship Between Gram and Squared Distance Matrices
  15. Proofs of Error Analysis of Non-Euclidean MDS
  16. ...and 7 more sections

Key Result

Theorem 2

It holds that $\lVert\hat{D}-D\rVert_F^2 = C_1+C_2+C_3$, where

Figures (3)

  • Figure 1: Negative (red), positive (blue).
  • Figure 2: Neuc-MDS and Neuc-MDS$^+$ consistently produce lower STRESS on all dimensions. Lower-MDS has a shorter curve because the target dimension $k$ is limited to be smaller than the number of positive eigenvalues.
  • Figure 3: Dimensionality Paradox on Eight Datasets.

Theorems & Definitions (22)

  • Definition 1: Non-Euclidean Dimension Reduction
  • Theorem 2
  • Theorem 3
  • Proposition 4: Semicircle Law wigner58
  • Theorem 5
  • Theorem 6
  • Remark 7
  • Proposition 8
  • Proposition 9
  • proof
  • ...and 12 more