Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck

TL;DR

An equivalence is established between a perfect measurement and a variant of the information bottleneck and this work can employ machine learning to optimize measurement processes that efficiently extract information from trajectory data.

Abstract

Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution with minimal redundancy. Finding an optimal measurement is challenging, and has generally required intimate knowledge of the dynamics in the few cases where it has been done. We establish an equivalence between a perfect measurement and a variant of the information bottleneck. As a consequence, we can employ machine learning to optimize measurement processes that efficiently extract information from trajectory data. We obtain approximately optimal measurements for multiple chaotic maps and lay the necessary groundwork for efficient information extraction from general time series.

Figures (5)

• Figure 1: (a) The attractor of the Ikeda map and a partial trajectory. States colored blue (orange) are measured as outcome A (B). The proposed method optimizes the measurement of a chaotic system using artificial neural networks that apply an information bottleneck (IB) and vector quantization (VQ) to each continuous-valued state. (b) The space of possible discrete measurements visualized in terms of the entropy of a single measurement, $H(U)$, and the rate of entropy production in the infinite duration limit, $h_\infty(U)$. The entropy rate of any partition is upper bounded by $H(U)$ and the metric entropy $h_\textnormal{KS}$ (dashed lines). The blue point corresponds to the partition in panel a that was optimized with the proposed method. The black points correspond to measurements parameterized by neural networks with random weights. Error bars on the entropy rates are within the markers.
• Figure 2: The fractional deviation of the entropy rate $h_\infty(U)$ from the metric entropy $h_\text{KS}$ as a function of the length of the trajectory $L$ used for training for (a) Ikeda and (b) Hénon maps with standard parameters, and (c) the logistic map with $r=3.7115$. The distribution of values of the entropy rate over 20 trials for each $L$ are shown as violin plots, with the extrema and median indicated by the black horizontal marks; the dotted line indicates $h_\infty(U)=0.99h_\text{KS}$. For each system, the partition found with the largest entropy rate for $L=12$ is shown on the right. For the logistic map, the probability density of states $p(x)$ is colored according to the optimized partition.
• Figure 3: (a) For optimized partitions of the Ikeda map, trained with sequences of length $L=12$ and with a base information annealing rate $\dot{\beta}=\dot{\beta}_0$, the reference state (i.e., the one predicted from the sequence of measurements) strongly influences the found partition. Each displayed partition is the one with the largest entropy rate (listed under the partition) after ten trials. Coordinate axes have been suppressed. (b) The same as in panel a, but with half the rate of information annealing $\dot{\beta}=0.5\dot{\beta}_0$ during training. (c) Two of the partitions of the Hénon map, identified with the same annealing rate as in panel a. (d) Same as in panel c for the logistic map with $r=3.7115$. (e) Colorings of the Ikeda attractor where, given a sequence of three measurements $\boldsymbol{u}_{:3}$, the reference state is the final state $x_3$, utilizing for the measurement $U$ the partition labelled (i) (left) and (ii) (right) in a and b, respectively.
• Figure S1: Estimating entropy rate.(a) The logistic map ($r=4$) generating partition with two symbols; $h_\textnormal{KS} = 1$ bit per iteration of the map. (b) The logistic map ($r=4$) generating partition with four symbols; $h_\textnormal{KS} = 1$ bit per iteration of the map. (c) The logistic map ($r=3.7115$) generating partition with two symbols; $h_\textnormal{KS} = 0.5203$ bit per iteration of the map. (d) The Hénon map ($a=1.4, b=0.3$); $h_\textnormal{KS} = 0.6048$ bits per iteration of the map. Error bars indicate the standard error from five repeats.
• Figure S2: Iterates of optimized partitions and established generating partitions. The partitions in the gray box, labelled timestep $n$, were iterated forward and backward three steps while retaining the coloring from timestep $n$. Coordinate axes have been suppressed. (a) The Ikeda map, with the partition at timestep $n$ the result of a run with $L=12$ and reference timestep $6/12$. (b) The Hénon map with standard parameters, optimized with the current method on the left and found by grassberger1985henon by threading certain homoclinic tangencies. (c) The logistic map with $r=3.7115$, optimized with the current method on the left and the known GP, with a boundary at the critical point at $x=0.5$ (true for any value of $r$). The invariant measure $p(x)$ is displayed vertically, and the bins are shaded according to the proportion of partition assignments at timestep $n$.