Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Tensor Train Approach for Deterministic Arithmetic Operations on Discrete Representations of Probability Distributions

Gerhard Kirsten, Bilgesu Bilgin, Janith Petangoda, Phillip Stanley-Marbell

TL;DR

The paper tackles the exponential blow-up in uncertainty propagation for high-dimensional probability distributions by introducing TT-TTR, a deterministic arithmetic framework that operates directly on discretized distributions represented in Telescoping Torques Dirac mixtures within a Tensor Train (TT) format. By updating TT cores in compressed space, TT-TTR achieves polynomial memory and computation under reasonable rank assumptions, and enables exact calculations of moments and covariances, avoiding stochastic Monte Carlo variability. The authors develop sparse TT-core implementations, correlation-aware alignment, and distribution-type caching to realize substantial memory and speed improvements over prior Dirac-mixure methods, demonstrated through problems in randomized linear algebra, integrated Brownian motion, Monte Carlo integration, and trace estimation. This approach offers a principled, reproducible alternative for high-dimensional uncertainty quantification and stochastic modeling, with potential applications in finance, engineering, and risk assessment where exact distributional arithmetic is crucial.

Abstract

Computing with discrete representations of high-dimensional probability distributions is fundamental to uncertainty quantification, Bayesian inference, and stochastic modeling. However, storing and manipulating such distributions suffers from the curse of dimensionality, as memory and computational costs grow exponentially with dimension. Monte Carlo methods require thousands to billions of samples, incurring high computational costs and producing inconsistent results due to stochasticity. We present an efficient tensor train method for performing exact arithmetic operations on discretizations of continuous probability distributions while avoiding exponential growth. Our approach leverages low-rank tensor train decomposition to represent latent random variables compactly using Dirac deltas, enabling deterministic addition, subtraction and multiplication operations directly in the compressed format. We develop an efficient implementation using sparse matrices and specialized data structures that further enhances performance. Theoretical analysis demonstrates polynomial scaling of memory and computational complexity under rank assumptions, and shows how statistics of latent variables can be computed with polynomial complexity. Numerical experiments spanning randomized linear algebra to stochastic differential equations demonstrate orders-of-magnitude improvements in memory usage and computational time compared to conventional approaches, enabling tractable deterministic computations on discretized random variables in previously intractable dimensions.

A Tensor Train Approach for Deterministic Arithmetic Operations on Discrete Representations of Probability Distributions

TL;DR

The paper tackles the exponential blow-up in uncertainty propagation for high-dimensional probability distributions by introducing TT-TTR, a deterministic arithmetic framework that operates directly on discretized distributions represented in Telescoping Torques Dirac mixtures within a Tensor Train (TT) format. By updating TT cores in compressed space, TT-TTR achieves polynomial memory and computation under reasonable rank assumptions, and enables exact calculations of moments and covariances, avoiding stochastic Monte Carlo variability. The authors develop sparse TT-core implementations, correlation-aware alignment, and distribution-type caching to realize substantial memory and speed improvements over prior Dirac-mixure methods, demonstrated through problems in randomized linear algebra, integrated Brownian motion, Monte Carlo integration, and trace estimation. This approach offers a principled, reproducible alternative for high-dimensional uncertainty quantification and stochastic modeling, with potential applications in finance, engineering, and risk assessment where exact distributional arithmetic is crucial.

Abstract

Computing with discrete representations of high-dimensional probability distributions is fundamental to uncertainty quantification, Bayesian inference, and stochastic modeling. However, storing and manipulating such distributions suffers from the curse of dimensionality, as memory and computational costs grow exponentially with dimension. Monte Carlo methods require thousands to billions of samples, incurring high computational costs and producing inconsistent results due to stochasticity. We present an efficient tensor train method for performing exact arithmetic operations on discretizations of continuous probability distributions while avoiding exponential growth. Our approach leverages low-rank tensor train decomposition to represent latent random variables compactly using Dirac deltas, enabling deterministic addition, subtraction and multiplication operations directly in the compressed format. We develop an efficient implementation using sparse matrices and specialized data structures that further enhances performance. Theoretical analysis demonstrates polynomial scaling of memory and computational complexity under rank assumptions, and shows how statistics of latent variables can be computed with polynomial complexity. Numerical experiments spanning randomized linear algebra to stochastic differential equations demonstrate orders-of-magnitude improvements in memory usage and computational time compared to conventional approaches, enabling tractable deterministic computations on discretized random variables in previously intractable dimensions.

Paper Structure

This paper contains 40 sections, 6 theorems, 32 equations, 5 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. An Illustrative Example
  3. The Status Quo in Uncertainty Quantification
  4. Discrete Representations of Probability Distributions
  5. Tensor Train Methods for High-Dimensional Distributions
  6. A New Finite-Dimensional Approach for High-Dimensional Distributions
  7. Contributions
  8. A review of the single- and multi-dimensional Dirac mixture representations
  9. Novel tensor train arithmetic framework
  10. Theoretical analysis and statistical computation
  11. Efficient implementation strategy and validation
  12. Deterministic arithmetic on probability distributions
  13. Notation and Mathematical Preliminaries
  14. Basic Operations on Random Variables
  15. Discrete Dirac Delta Representation of Random Variables
  16. ...and 25 more sections

Key Result

Theorem 1

Let $X_1, \ldots, X_d$ be independent random variables. The sum $Z = \sum_{k=1}^d X_k$ admits tensor train representations with support tensor rank 2 and probability mass tensor rank 1: where the support cores $\mathcal{S}^{(k)}$ have slices: and the mass cores $\mathcal{M}^{(k)}$ have slices:

Figures (5)

  • Figure 1: Matrix and tensor structures resulting from (left) multiplication of two and (right) addition of three independent TTR-discretized distributions.
  • Figure 1: Tensor train decompositions: (left) applying Theorem \ref{['thm:tt_multiplication']} to the multiplication in Figure \ref{['fig:randomMatrix']}, left; (right) applying Theorem \ref{['thm:tt_addition']} to the addition in Figure \ref{['fig:randomMatrix']}, right.
  • Figure 1: Performance comparison between jTTR and TT-TTR implementations. Top row: Computation time for addition (left) and multiplication (right). Bottom row: Memory usage for addition (left) and multiplication (right). Right column: Speedup factor as dimensions increase.
  • Figure 2: Probability density functions of $W_T$, $I_1$, and $I_2$ from the TT-TTR approach with $n=64$ and $n=1024$. Histograms are constructed by binning the discrete support points and their masses.
  • Figure 3: Probability density functions of the three outputs from Monte Carlo with $s=1000$ and $s=500000$ paths.

Theorems & Definitions (20)

  • Definition 1: Discrete Convolution
  • Definition 2: Product Distribution
  • Theorem 1: Addition in Tensor Train Format
  • Proof 1
  • Theorem 2: Multiplication in Tensor Train Format
  • Proof 2
  • Theorem 3: General Tensor Train Arithmetic
  • Proof 3
  • Theorem 4: Tensor Train Alignment with Correlation Preservation
  • Proof 4
  • ...and 10 more