Dynamic Structure Estimation from Bandit Feedback using Nonvanishing Exponential Sums

TL;DR

This work tackles the dynamic structure estimation problems for periodically behaved discrete dynamical system in the Euclidean space with efficient algorithms that exploit asymptotic behaviors of exponential sums to effectively average out the noise effect while preventing the information to be estimated from vanishing.

Abstract

This work tackles the dynamic structure estimation problems for periodically behaved discrete dynamical system in the Euclidean space. We assume the observations become sequentially available in a form of bandit feedback contaminated by a sub-Gaussian noise. Under such fairly general assumptions on the noise distribution, we carefully identify a set of recoverable information of periodic structures. Our main results are the (computation and sample) efficient algorithms that exploit asymptotic behaviors of exponential sums to effectively average out the noise effect while preventing the information to be estimated from vanishing. In particular, the novel use of the Weyl sum, a variant of exponential sums, allows us to extract spectrum information for linear systems. We provide sample complexity bounds for our algorithms, and we experimentally validate our theoretical claims on simulations of toy examples, including Cellular Automata.

Figures (9)

• Figure 1: Overview of how the exponential sum techniques are used in the work. For (nearly) period estimation problem, it is an application of discrete Fourier transform in our statistical settings, which ensures that the correct estimate remains while noise effect or wrong estimates are properly suppressed. When the system follows linear dynamics in addition to the (nearly) periodicity, our application of the Weyl sum, a variant of exponential sums, preserves some set of the eigenstructure information.
• Figure 2: Illustration of loss of dynamic information through concentration of measure. Left: An example of the law of large numbers showing the sample mean of the numbers given by throwing dice many times will converge towards the expected value as the number of throws increases. Right: An intuitive understanding of how the concentration of measures or process of averaging out the observations may erase not only the noise effect but also the dynamics information; in this case, the dynamics on the left side is a fixed point attractor and the right one is a limit cycle attractor, both of which may return $(0,0)$ when averaging the states over long time.
• Figure 3: Illustration of the example practical case where our techniques may be applied; in this example, the target object is following nearly periodic motion, and requires concentration of light to be captured. A random filter could be employed and we observe a single value sequentially to predict its periodic information.
• Figure 4: Illustration of the position of our work as a study of reconstruction of dynamical system properties. We assume sub-Gaussian noise, bandit feedback (user-defined linear feedback contaminated by noise), autonomous system, and (nearly) periodicity which is the information to be extracted. To our knowledge, our assumptions do not subsume or are subsumed by those of other work as a problem of provably estimating periodic structure. In comparison to existing system identification techniques for partially observed dynamical systems, we do not assume Gaussian (or bounded) noise, limited budget for adversarial noise, controllability, observability or noisy transition, while we assume bandit feedback to identify a recoverable set of dynamic structure information.
• Figure 5: Left: Illustration of a period eight instance of LifeGame; (top) original transitions. (down) an instance of noisy observation. Right: $\mu$-nearly periodic dynamics \ref{['muperiodicex']}.

Theorems & Definitions (46)

• Definition 4.1: Nearly periodic sequence
• Definition 4.2: Aliquot nearly period
• Example 4.1
• Proposition 4.1
• proof
• Proposition 4.2
• proof
• Corollary 4.3
• proof
• Example 4.2