Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Position: Tensor Networks are a Valuable Asset for Green AI

Eva Memmel, Clara Menzen, Jetze Schuurmans, Frederiek Wesel, Kim Batselier

TL;DR

The paper tackles the unsustainable growth of AI compute by linking Tensor Networks (TNs) with Green AI to achieve efficiency without sacrificing performance. It surveys Green AI metrics and demonstrates how TNs—via CP, Tucker, and TT decompositions—enable logarithmic compression and scalable, low-rank representations. Through kernel machines and deep learning, the authors show concrete efficiency gains and occasional accuracy trade-offs, underscoring TNs as a practical component of a broader Green AI portfolio. The work also discusses limitations, such as data correlations and infrastructure impacts, and outlines future research directions to foster sustainable and inclusive AI progress through TN adoption.

Abstract

For the first time, this position paper introduces a fundamental link between tensor networks (TNs) and Green AI, highlighting their synergistic potential to enhance both the inclusivity and sustainability of AI research. We argue that TNs are valuable for Green AI due to their strong mathematical backbone and inherent logarithmic compression potential. We undertake a comprehensive review of the ongoing discussions on Green AI, emphasizing the importance of sustainability and inclusivity in AI research to demonstrate the significance of establishing the link between Green AI and TNs. To support our position, we first provide a comprehensive overview of efficiency metrics proposed in Green AI literature and then evaluate examples of TNs in the fields of kernel machines and deep learning using the proposed efficiency metrics. This position paper aims to incentivize meaningful, constructive discussions by bridging fundamental principles of Green AI and TNs. We advocate for researchers to seriously evaluate the integration of TNs into their research projects, and in alignment with the link established in this paper, we support prior calls encouraging researchers to treat Green AI principles as a research priority.

Position: Tensor Networks are a Valuable Asset for Green AI

TL;DR

The paper tackles the unsustainable growth of AI compute by linking Tensor Networks (TNs) with Green AI to achieve efficiency without sacrificing performance. It surveys Green AI metrics and demonstrates how TNs—via CP, Tucker, and TT decompositions—enable logarithmic compression and scalable, low-rank representations. Through kernel machines and deep learning, the authors show concrete efficiency gains and occasional accuracy trade-offs, underscoring TNs as a practical component of a broader Green AI portfolio. The work also discusses limitations, such as data correlations and infrastructure impacts, and outlines future research directions to foster sustainable and inclusive AI progress through TN adoption.

Abstract

For the first time, this position paper introduces a fundamental link between tensor networks (TNs) and Green AI, highlighting their synergistic potential to enhance both the inclusivity and sustainability of AI research. We argue that TNs are valuable for Green AI due to their strong mathematical backbone and inherent logarithmic compression potential. We undertake a comprehensive review of the ongoing discussions on Green AI, emphasizing the importance of sustainability and inclusivity in AI research to demonstrate the significance of establishing the link between Green AI and TNs. To support our position, we first provide a comprehensive overview of efficiency metrics proposed in Green AI literature and then evaluate examples of TNs in the fields of kernel machines and deep learning using the proposed efficiency metrics. This position paper aims to incentivize meaningful, constructive discussions by bridging fundamental principles of Green AI and TNs. We advocate for researchers to seriously evaluate the integration of TNs into their research projects, and in alignment with the link established in this paper, we support prior calls encouraging researchers to treat Green AI principles as a research priority.
Paper Structure (13 sections, 6 equations, 4 figures, 1 table)

This paper contains 13 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Green AI in Related Literature
  3. The Negative Impact of Growing Compute on Sustainability and AI Progress
  4. Efficiency Metrics Suggested in Literature
  5. Essentials of Tensor Networks for Green AI
  6. From Vectors and Matrices to Tensors and then Tensor Networks
  7. Commonly Used Tensor Networks
  8. Logarithmic Compression Potential of Low-Rank Tensor Networks as an Enabler for Green AI
  9. Applications of Tensor Networks to AI
  10. Tensor Networks in Kernel Machines
  11. Tensor Networks in Deep Learning
  12. Green AI Analysis of Selected Tensor Network Models with Regard to Efficiency Metrics
  13. Critical Discussion and Conclusion

Figures (4)

  • Figure 1: This position paper underscores the importance of connecting the research fields of TNs and Green AI. By establishing this link for the first time, we aim to achieve two primary goals. Firstly, to encourage the TN community to embrace Green AI practices and consciously tailor their TN models for enhanced sustainability. Secondly, to motivate the AI community to adopt Green AI practices and to explore the integration of TNs in their research. We believe that the combined application of Green AI practices and TNs will not only foster the social, economic, and ecological sustainability of AI models but also enhance the inclusivity and diversity of AI research. Ultimately, this synergy is expected to amplify the positive impact of AI research as a whole.
  • Figure 2: Left: tensorization of a matrix of size $2^3\times2^3$ into a sixth-order tensor of size $2\times2\times2\times2\times2\times2$. The two crosses illustrate the organization of the entries. Based on Fig. 2 of cichocki2015tensor. Right: tensorization in diagram notation. The 2 edges of the node representing the 8-by-8 matrix become 6 edges sticking out the node representing the sixth-order tensor.
  • Figure 3: Graphical depiction of commonly used TNs for a third-order tensor. Connected edges are indices that are being summed over. The CP decomposition is a special case of Tucker, where the core tensor is diagonal. This is shown by the diagonal in the node.
  • Figure 4: Demonstrating the impact of Tucker, TT, and CP decompositions on a $I^D$ tensor with $I=100$ and $R=10$. Without decomposition, the tensor's storage complexity increases exponentially with $D$. The Tucker decomposition yields a slower exponential growth, while CP and TT decompositions grow linear in $D$.