Universal consistency of the $k$-NN rule in metric spaces and Nagata dimension. III
Vladimir G. Pestov
TL;DR
The paper proves that a complete separable metric space not sigma-finite dimensional in the Nagata sense precludes universal $k$-NN consistency, by constructing a measure $\mu$ and regression $\eta$ that induce convergence to the wrong label along a subsequence. It complements known equivalences (3)⇔(2) from Preiss and (2)⇒(1) from Cérou–Guyader, showing (1)⇒(3) as the remaining implication, thus completing the trilogy of conditions. The work uses a intricate measure construction (generalizing Preiss) on a compact set $K$ built from an indexing tree to create a dichotomy between atomic and diffuse parts, yielding a counterexample to universal consistency. It also corrects a misunderstanding about the sufficiency of the weak Lebesgue--Besicovitch property by highlighting the Heisenberg group as a counterexample, where the weak property holds but Nagata-dimension fails, thus breaking universal consistency.
Abstract
We prove the last remaining implication allowing to claim the equivalence of the following conditions for a complete separable metric space $X$: (1) The $k$-nearest neighbour classifier is (weakly) universally consistent in $X$, (2) The strong Lebesgue--Besicovitch differentiation property holds in $X$ for every locally finite Borel measure, (3) $X$ is sigma-finite dimensional in the sense of Nagata. The equivalence (2)$\iff$(3) was announced by Preiss (1983), while a detailed proof of the implication (3)$\Rightarrow$(2) has appeared in Assouad and Quentin de Gromard (2006). The implication (2)$\Rightarrow$(1) was established by Cérou and Guyader (2006). We prove the implication (1)$\Rightarrow$(3). The result was conjectured in the first article in the series (Collins, Kumari, Pestov 2020), and here we also correct a wrong claim made in the second article (Kumari and Pestov 2024).