Measure Theory of Conditionally Independent Random Function Evaluation
Felix Benning
TL;DR
The paper develops a rigorous measure-theoretic framework for evaluating random functions at random inputs, addressing when previsible or conditionally independent inputs can be treated as deterministic in conditional calculations. It introduces joint probability kernels for collections of conditional distributions, proves existence and continuity of these kernels under suitable regularity, and provides Gaussian-process special cases with explicit, continuous forms. By relating $\mathbb{E}[f(X)|F]$ to a measurable function $H$ and establishing continuity in the index, the work justifies common heuristics used in Bayesian optimization, Kriging, and related fields, including scenarios with noisy observations. A topological foundation ensures the evaluation map $e(f,x)=f(x)$ is measurable on $C(X,Y)$ via the compact-open topology, clarifying when these constructions are well-defined and highlighting limitations when $X$ lacks local compactness.
Abstract
The next evaluation point $x_{n+1}$ of a random function $\mathbf f = (\mathbf f(x))_{x\in \mathbb X}$ (a.k.a. stochastic process or random field) is often chosen based on the filtration of previously seen evaluations $\mathcal F_n := σ(\mathbf f(x_0),\dots, \mathbf f(x_n))$. This turns $x_{n+1}$ into a random variable $X_{n+1}$ and thereby $\mathbf f(X_{n+1})$ into a complex measure theoretical object. In applications, like geostatistics or Bayesian optimization, the evaluation locations $X_n$ are often treated as deterministic during the calculation of the conditional distribution $\mathbb P(\mathbf f(X_{n+1}) \in A \mid \mathcal F_n)$. We provide a framework to prove that the results obtained by this treatment are typically correct. We also treat the more general case where $X_{n+1}$ is not 'previsible' but independent from $\mathbf f$ conditional on $\mathcal F_n$ and the case of noisy evaluations.