Non-Asymptotic Error Bounds for Causally Conditioned Directed Information Rates of Gaussian Sequences

TL;DR

An explicit formula for causally conditioned directed information rate based on optimal prediction is provided and an estimator based on this formula is defined, showing that the estimator gives an error of order $O\left(N^{-1/2}\log(N)$ with high probability, where $N$ is the total sample size.

Abstract

Directed information and its causally conditioned variations are often used to measure causal influences between random processes. In practice, these quantities must be measured from data. Non-asymptotic error bounds for these estimates are known for sequences over finite alphabets, but less is known for real-valued data. This paper examines the case in which the data are sequences of Gaussian vectors. We provide an explicit formula for causally conditioned directed information rate based on optimal prediction and define an estimator based on this formula. We show that our estimator gives an error of order $O\left(N^{-1/2}\log(N)\right)$ with high probability, where $N$ is the total sample size.