Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection

Giorgos Iacovides, Wuyang Zhou, Danilo Mandic

TL;DR

A novel framework that leverages large language models (LLMs) to guide the rank selection in tensor network models for higher-order data analysis and offers enhanced interpretability of the rank choices and can effectively optimise the objective function is proposed.

Abstract

We propose a novel framework that leverages large language models (LLMs) to guide the rank selection in tensor network models for higher-order data analysis. By utilising the intrinsic reasoning capabilities and domain knowledge of LLMs, our approach offers enhanced interpretability of the rank choices and can effectively optimise the objective function. This framework enables users without specialised domain expertise to utilise tensor network decompositions and understand the underlying rationale within the rank selection process. Experimental results validate our method on financial higher-order datasets, demonstrating interpretable reasoning, strong generalisation to unseen test data, and its potential for self-enhancement over successive iterations. This work is placed at the intersection of large language models and higher-order data analysis.

Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection

TL;DR

A novel framework that leverages large language models (LLMs) to guide the rank selection in tensor network models for higher-order data analysis and offers enhanced interpretability of the rank choices and can effectively optimise the objective function is proposed.

Abstract

We propose a novel framework that leverages large language models (LLMs) to guide the rank selection in tensor network models for higher-order data analysis. By utilising the intrinsic reasoning capabilities and domain knowledge of LLMs, our approach offers enhanced interpretability of the rank choices and can effectively optimise the objective function. This framework enables users without specialised domain expertise to utilise tensor network decompositions and understand the underlying rationale within the rank selection process. Experimental results validate our method on financial higher-order datasets, demonstrating interpretable reasoning, strong generalisation to unseen test data, and its potential for self-enhancement over successive iterations. This work is placed at the intersection of large language models and higher-order data analysis.

Paper Structure

This paper contains 21 sections, 2 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Tensor Network Rank Search and Its Search Space
  3. Objective Function.
  4. LLM-guided Interpretable Rank Selection
  5. System Message.
  6. Initial Prompt.
  7. Iterative Prompt.
  8. Methodology
  9. Early Stopping.
  10. Rank Constraints.
  11. Interpretability.
  12. Experimental Results
  13. Data Preparation.
  14. Experimental Settings.
  15. Quantitative Results.
  16. ...and 6 more sections

Figures (3)

  • Figure 1: Setting a rank in the FCTN topology to $1$ is equivalent to dropping that connection, resulting in a new tensor network structure. In this way, tensor network rank search for FCTN becomes equivalent to tensor network structure search under the constraint of a complete graph.
  • Figure 2: The proposed framework for LLM-guided tensor network rank search. The LLM first uses domain knowledge to reason about the mode interactions and then suggests the tensor network ranks. After obtaining the objective function values, the LLM tries to revise its reasoning and proposes a new set of tensor network ranks to optimise the objective function in an iterative cycle.
  • Figure 3: Visualisation of the experimental results; Left: Comparison of the objective function between our proposed framework and the baseline models across 10 iterations. Right: Evolution of the objective function for our proposed framework across 10 iterations with the line of best fit.