On three papers by Jurgens \& Crutchfield, and on the basic structure of "computational mechanics"

TL;DR

An alternative complexity measure for models where the forecasting complexity is infinite is proposed, and it is pointed out that the results apply also beyond hidden Markov models.

Abstract

In a recent paper, Jurgens and Crutchfield [Phys. Rev. E {\bf 104}, 064107 (2021), called ``paper III" in the following] computed what they called the ``ambiguity rate" of hidden Markov processes, a concept supposedly introduced by Claude Shannon. This calculation was based on a ``mixed state" formalism introduced by them in J. Stat. Phys. {\bf 183}, 32 (2021) (``paper I"), and developed further in Chaos, {\bf 31}, 083114 (2021) (``paper II"). We point out that (i) ambiguity rates were {\it not} introduced by Shannon; (ii) their computations in paper III are wrong, because of an error made already in papers I and II; (iii) due to this error (a confusion between open sets and their closures), also many of the ``statistical complexity dimensions" computed in II are wrong; (iv) the ``mixed state" formalism of I is just the well known `forward algorithm' for hidden Markov models; (v) the `causal states' in `$ε$-machines' correspond in general to {\it finite} (as opposed to infinite, as often claimed) histories; and (vi) `$ε$-machines' are always countable, in contrast to frequent claims in the literature. In addition, we propose an alternative complexity measure for models where the forecasting complexity is infinite, and we point out that our results apply also beyond hidden Markov models.