Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)

Junhua Zeng, Chao Li, Zhun Sun, Qibin Zhao, Guoxu Zhou

TL;DR

This work tackles TN-SS, the problem of selecting TN structure hyperparameters, by introducing tnGPS, an LLM-driven framework that mimics human research workflows to automatically discover new TN-SS algorithms. By modeling the innovation process through prompts for knowledge categorization, recombination, incremental improvements, and diversity injection, tnGPS generates candidates that are evaluated on downstream tasks and iteratively refined. Experimental results on natural image and Gaussian process model compression show that tnGPS-discovered TN-SS algorithms systematically outperform state-of-the-art methods, including in out-of-domain settings, with ablations confirming the value of KR, II, and DI components. The approach demonstrates the potential of using large language models to automate algorithm discovery, offering a scalable path to improve TN-based representations while highlighting practical considerations like prompt design and model variability.

Abstract

Tensor networks are efficient for extremely high-dimensional representation, but their model selection, known as tensor network structure search (TN-SS), is a challenging problem. Although several works have targeted TN-SS, most existing algorithms are manually crafted heuristics with poor performance, suffering from the curse of dimensionality and local convergence. In this work, we jump out of the box, studying how to harness large language models (LLMs) to automatically discover new TN-SS algorithms, replacing the involvement of human experts. By observing how human experts innovate in research, we model their common workflow and propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement. The experimental results demonstrate that the algorithms discovered by tnGPS exhibit superior performance in benchmarks compared to the current state-of-the-art methods.

tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)

TL;DR

This work tackles TN-SS, the problem of selecting TN structure hyperparameters, by introducing tnGPS, an LLM-driven framework that mimics human research workflows to automatically discover new TN-SS algorithms. By modeling the innovation process through prompts for knowledge categorization, recombination, incremental improvements, and diversity injection, tnGPS generates candidates that are evaluated on downstream tasks and iteratively refined. Experimental results on natural image and Gaussian process model compression show that tnGPS-discovered TN-SS algorithms systematically outperform state-of-the-art methods, including in out-of-domain settings, with ablations confirming the value of KR, II, and DI components. The approach demonstrates the potential of using large language models to automate algorithm discovery, offering a scalable path to improve TN-based representations while highlighting practical considerations like prompt design and model variability.

Abstract

Tensor networks are efficient for extremely high-dimensional representation, but their model selection, known as tensor network structure search (TN-SS), is a challenging problem. Although several works have targeted TN-SS, most existing algorithms are manually crafted heuristics with poor performance, suffering from the curse of dimensionality and local convergence. In this work, we jump out of the box, studying how to harness large language models (LLMs) to automatically discover new TN-SS algorithms, replacing the involvement of human experts. By observing how human experts innovate in research, we model their common workflow and propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement. The experimental results demonstrate that the algorithms discovered by tnGPS exhibit superior performance in benchmarks compared to the current state-of-the-art methods.
Paper Structure (21 sections, 3 equations, 12 figures, 5 tables, 1 algorithm)

This paper contains 21 sections, 3 equations, 12 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Tensor Network Structure Search (TN-SS)
  5. A Unified Paradigm for TN-SS Algorithms
  6. How Human Experts do Innovation?
  7. tnGPS: a LLM-Driven Framework for TN-SS Algorithm discovery
  8. Experimental Results
  9. Natural Images Compression
  10. Model Compression for Gaussian Process: an Out-of-Domain Experiment
  11. Ablations
  12. The workflow of tnGPS
  13. Codes for different TN-SS algorithms
  14. Code for Ho-1
  15. Code for Ho-2
  16. ...and 6 more sections

Figures (12)

  • Figure 1: A basic workflow to illustrate how human experts do innovation in research.
  • Figure 2: The prompt used for interface description.
  • Figure 3: The key prompt used in knowledge categorisation (KC). The purple sentence specifies the goal of the prompt.
  • Figure 4: The key prompt used in knowledge recombination (KR). The purple sentence specifies the goal and the orange sentence clarifies the restriction.
  • Figure 5: The key prompt used in incremental innovation (II). The purple sentence specifies the goal and the orange sentence clarifies the restriction.
  • ...and 7 more figures