Learning Transfer Operators by Kernel Density Estimation

TL;DR

This work reframes the estimation of the Frobenius-Perron transfer operator from dynamical systems as a density-estimation problem. By interpreting Ulam-Galerkin as a histogram estimator for conditional densities, the authors develop a KDE-based framework that analyzes bias, variance, and mean squared error, showing KDE often yields higher accuracy than histogram approaches. They derive theoretical results for both histogram and KDE estimators, establish optimal bandwidths, and validate the approach with logistic and Markov-map examples, including estimations of $\rho(x)$, $\rho(x,x')$, and $\rho(x'|x)$ and the associated operator $P$. The findings highlight the potential of KDE and other density-estimation methods to improve finite-sample estimates of transfer operators, while also noting boundary effects and opportunities for high-dimensional extensions.

Abstract

Inference of transfer operators from data is often formulated as a classical problem that hinges on the Ulam method. The conventional description, known as the Ulam-Galerkin method, involves projecting onto basis functions represented as characteristic functions supported over a fine grid of rectangles. From this perspective, the Ulam-Galerkin approach can be interpreted as density estimation using the histogram method. In this study, we recast the problem within the framework of statistical density estimation. This alternative perspective allows for an explicit and rigorous analysis of bias and variance, thereby facilitating a discussion on the mean square error. Through comprehensive examples utilizing the logistic map and a Markov map, we demonstrate the validity and effectiveness of this approach in estimating the eigenvectors of the Frobenius-Perron operator. We compare the performance of Histogram Density Estimation(HDE) and Kernel Density Estimation(KDE) methods and find that KDE generally outperforms HDE in terms of accuracy. However, it is important to note that KDE exhibits limitations around boundary points and jumps. Based on our research findings, we suggest the possibility of incorporating other density estimation methods into this field and propose future investigations into the application of KDE-based estimation for high-dimensional maps. These findings provide valuable insights for researchers and practitioners working on estimating the Frobenius-Perron operator and highlight the potential of density estimation techniques in this area of study. Keywords: Transfer Operators; Frobenius-Perron operator; probability density estimation; Ulam-Galerkin method; Kernel Density Estimation; Histogram Density Estimation.

Figures (17)

• Figure 1: The figure presents data consisting of $N=1,000$ samples of $(x_n,x_{n+1})$ pairs, obtained from two different systems. The left panel shows a simulation obtained from the logistic map $x_{n+1}=f(x_n)=4 x_n (1-x_n)$, after an initial transient, so that the sample distribution closely approximates the invariant distribution $\rho_X(x)=\frac{1}{\pi\sqrt{x(1-x)}}$. On the right panel, the data is obtained from a noisy logistic map $x_{n+1}=f(x_n)=4 x_n (1-x_n)+s_n$, where $s_n$ is chosen from an independent and identically distributed (i.i.d.) truncated normal distribution with standard deviation $\sigma=0.02$.
• Figure 2: This figure displays the kernel function ($\nu$) of the Frobenius-Perron operator for the truncated normal distribution, as described in \ref{['tnormal']}. The figure illustrates the kernel for three different standard deviations, namely $\sigma=0.025$, $\sigma=0.05$, and $\sigma=0.1$. The "bumps" observed in the distribution are a direct consequence of the requirement for a bounded domain, which ensures that the distribution remains a probability distribution with a unity integral. Additional insights can be obtained by referring to the sample data presented in \ref{['fig1']}.
• Figure 3: The figure shows the for data $(x_n,x_{n+1})$ from sample orbits of size $N=1,000$ (top row) and $N=10,000$ (bottom row) using the logistic map (noise-free). The bin-width used for estimation is $1/K$ with $K=40$, and the joint distribution estimate employs a grid of $40 \times 40$ cells. The bottom row, which represents higher sample orbits, provides smoother and more accurate estimates of the true distributions with reduced variability. Comparing the estimated marginal distributions to the true distribution (left figures) allows for insights into the effectiveness of the estimation techniques employed in this study. The relatively higher densities observed around points $(0,0)$, $(0,1)$, and $(0.5,1)$ in the middle and right figures indicate changes in data concentration from $x_n$ to $x_{n+1}$.
• Figure 4: The data in this figure is similar to that presented in \ref{['fig2']}, but it uses with a coarser resolution. The estimation employs $K=10$ and $10 \times 10$ cells for marginal, joint, and conditional densities, resulting in a wider bandwidth that reduces variability but increases bias.
• Figure 5: The figure showcases the , similar to \ref{['fig4']} using data from a random Logistic map orbit with truncated normal distribution noise ($\sigma=0.02$) as depicted in \ref{['fig1']}. These plots exhibit similar characteristics of variability versus bias (smoothing) as observed in the noiseless scenario of \ref{['fig2']}(Top Row), even with the same level of smoothing, despite the differences in the true underlying distribution. However, it is noteworthy that the marginal distribution estimate of the invariant distribution appears significantly less smooth.