On hypoellipticity of degenerate operators in testing and detection problems
Erhan Bayraktar, Yuqiong Wang
TL;DR
The paper addresses hypoellipticity for degenerate diffusion operators arising in filtering-based sequential testing and quickest detection under partial information. It develops a posterior-coordinate framework and derives a sum-of-squares representation of the generator, enabling a Lie-bracket analysis of Hörmander’s condition. For the testing case ($Q=0$), a sharp criterion ties hypoellipticity to the drift-difference matrix and the squared-drift-norm vector, with automatic failure when $n>k+1$. For the detection case ($Q\neq 0$), two practical sufficient conditions are provided to guarantee hypoellipticity, along with a parabolic extension showing time-dependent problems inherit regularity. The results yield smoothing and strong-Feller properties for the posterior process and regularity of the value function within continuation regions, illustrated by several multi-dimensional examples with partial information and regime-switching dynamics.
Abstract
We study a class of degenerate diffusion generators that arise in sequential testing and quickest detection problems with partial information. The observation process is driven by $k$ independent Brownian motions, while the hidden state takes $n+1$ values with $k<n$. By moving to the posterior likelihood coordinates, we analyze the Hömander's condition of the operator both without state switching (testing) and with switching (detection). We characterize the cases where the operator is hypoelliptic for the former, give two different sufficient conditions for the latter, and discuss their consequences.