Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparse Domain Transfer via Elastic Net Regularization

Jingwei Zhang, Farzan Farnia

TL;DR

The paper tackles sparse domain transfer by introducing Elastic Net Optimal Transport (ENOT), which regularizes the OT cost with both $L_1$ and $L_2$ terms to produce sparse transport maps. It derives a dual formulation with a Brenier-type theorem showing that the optimal transport can be obtained via a soft-thresholded gradient of the dual optimal potential, enabling controlled sparsity through the elastic-net parameter. A feature-selection-based extension further reduces the problem to unconstrained transport on a selected feature subset, implemented via a GAN framework. Empirical results across synthetic Gaussian mixtures, image translation tasks, and IMDB sentiment reversal demonstrate that ENOT yields meaningful sparse transports, identifiable salient regions/words, and competitive transfer quality with reduced modifications. The work thereby provides a flexible, interpretable approach to sparse domain transfer with broad applicability across vision and NLP.

Abstract

Transportation of samples across different domains is a central task in several machine learning problems. A sensible requirement for domain transfer tasks in computer vision and language domains is the sparsity of the transportation map, i.e., the transfer algorithm aims to modify the least number of input features while transporting samples across the source and target domains. In this work, we propose Elastic Net Optimal Transport (ENOT) to address the sparse distribution transfer problem. The ENOT framework utilizes the $L_1$-norm and $L_2$-norm regularization mechanisms to find a sparse and stable transportation map between the source and target domains. To compute the ENOT transport map, we consider the dual formulation of the ENOT optimization task and prove that the sparsified gradient of the optimal potential function in the ENOT's dual representation provides the ENOT transport map. Furthermore, we demonstrate the application of the ENOT framework to perform feature selection for sparse domain transfer. We present the numerical results of applying ENOT to several domain transfer problems for synthetic Gaussian mixtures and real image and text data. Our empirical results indicate the success of the ENOT framework in identifying a sparse domain transport map.

Sparse Domain Transfer via Elastic Net Regularization

TL;DR

The paper tackles sparse domain transfer by introducing Elastic Net Optimal Transport (ENOT), which regularizes the OT cost with both and terms to produce sparse transport maps. It derives a dual formulation with a Brenier-type theorem showing that the optimal transport can be obtained via a soft-thresholded gradient of the dual optimal potential, enabling controlled sparsity through the elastic-net parameter. A feature-selection-based extension further reduces the problem to unconstrained transport on a selected feature subset, implemented via a GAN framework. Empirical results across synthetic Gaussian mixtures, image translation tasks, and IMDB sentiment reversal demonstrate that ENOT yields meaningful sparse transports, identifiable salient regions/words, and competitive transfer quality with reduced modifications. The work thereby provides a flexible, interpretable approach to sparse domain transfer with broad applicability across vision and NLP.

Abstract

Transportation of samples across different domains is a central task in several machine learning problems. A sensible requirement for domain transfer tasks in computer vision and language domains is the sparsity of the transportation map, i.e., the transfer algorithm aims to modify the least number of input features while transporting samples across the source and target domains. In this work, we propose Elastic Net Optimal Transport (ENOT) to address the sparse distribution transfer problem. The ENOT framework utilizes the -norm and -norm regularization mechanisms to find a sparse and stable transportation map between the source and target domains. To compute the ENOT transport map, we consider the dual formulation of the ENOT optimization task and prove that the sparsified gradient of the optimal potential function in the ENOT's dual representation provides the ENOT transport map. Furthermore, we demonstrate the application of the ENOT framework to perform feature selection for sparse domain transfer. We present the numerical results of applying ENOT to several domain transfer problems for synthetic Gaussian mixtures and real image and text data. Our empirical results indicate the success of the ENOT framework in identifying a sparse domain transport map.
Paper Structure (19 sections, 3 theorems, 35 equations, 7 figures, 6 tables)

This paper contains 19 sections, 3 theorems, 35 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Elastic Net Regularization for Sparse Optimal Transport
  5. ENOT-based Feature Selection for Sparse Domain Transfer
  6. Numerical Results
  7. ENOT applied to Synthetic Gaussian Mixture Data
  8. Image-based Domain Transfer
  9. IMDB Review Sentiment Reversal
  10. Conclusion
  11. Appendix
  12. Proofs
  13. Proof of Theorem \ref{['Thm: Weak Concavity: ENOT potential']}
  14. Proof of Theorem \ref{['Theorem: Brenier, ENOT']}
  15. Experimental Setting
  16. ...and 4 more sections

Key Result

Theorem 1

Suppose that $P_X,P_Y$ are absolutely continuous with respect to one another. Then, the gradient of the solution $\widetilde{\phi}^*$ to equation Eq: OT 2-Wasserstein dual is the unique monotone map for transferring $P_X$ to $P_Y$, that is

Figures (7)

  • Figure 1: Transportation for Black$\rightarrow$Blonde hair and Apple$\rightarrow$Orange on CelebA and Apple2Orange.
  • Figure 2: Saliency maps of transportation for Black$\rightarrow$Blonde hair and Apple$\rightarrow$Orange on CelebA and Apple2Orange.
  • Figure 3: Left: IMDB sentiment transfer quality and density. Every point represents a transported sample: color indicates the $L_1$-coefficient in ENOT. The x vs. y axes are the transportation quality vs. density. Right: Transportation quality, the middle line and diamond show the median and mean. The green bar (coefficient $1e^{-4}$) achieves the highest score.
  • Figure 4: Source and transportation maps for Apple$\rightarrow$Orange dataset.
  • Figure 5: Source and transportation maps for Black$\rightarrow$Blonde hair on CelebA dataset.
  • ...and 2 more figures

Theorems & Definitions (7)

  • Example 1
  • Example 2
  • Theorem 1: Brenier's Theorem, villani2009optimal
  • Theorem 2
  • proof
  • Theorem 3
  • proof