Recursively Enumerably Representable Classes and Computable Versions of the Fundamental Theorem of Statistical Learning
David Kattermann, Lothar Sebastian Krapp
TL;DR
This work addresses how computable PAC (CPAC) learning alters the classical Fundamental Theorem of Statistical Learning, introducing the effective VC-dimension $eVCdim$ and recursively enumerable representable (RER) classes to bridge computability with learnability. It proves that $VCdim$ and $eVCdim$ can diverge for RER classes (construction of $\text{H}^{k,\ell}$ with $VCdim=k$ and $eVCdim=\ell$), yet they coincide under strong CPAC assumptions when a total computable ERM exists. A central result shows that CPAC learnability can be characterized by containment of an RER subclass that realizes the same samples, with UIP yielding a partial converse linking RER structure to realizable CPAC learnability. Additionally, nonuniform CPAC learning is shown to hold for all RER classes via structural risk minimization, highlighting a computable, robust path to learning without requiring finite $eVCdim$. These insights illuminate the limits and possibilities of recovering fundamental learning principles in the computable regime and raise questions about the role of uncomputable hypotheses and realizable CPAC characterizations.
Abstract
We study computable probably approximately correct (CPAC) learning, where learners are required to be computable functions. It had been previously observed that the Fundamental Theorem of Statistical Learning, which characterizes PAC learnability by finiteness of the Vapnik-Chervonenkis (VC-)dimension, no longer holds in this framework. Recent works recovered analogs of the Fundamental Theorem in the computable setting, for instance by introducing an effective VC-dimension. Guided by this, we investigate the connection between CPAC learning and recursively enumerable representable (RER) classes, whose members can be algorithmically listed. Our results show that the effective VC-dimensions can take arbitrary values above the traditional one, even for RER classes, which creates a whole family of (non-)examples for various notions of CPAC learning. Yet the two dimensions coincide for classes satisfying sufficiently strong notions of CPAC learning. We then observe that CPAC learnability can also be characterized via containment of RER classes that realize the same samples. Furthermore, it is shown that CPAC learnable classes satisfying a unique identification property are necessarily RER. Finally, we establish that agnostic learnability can be guaranteed for RER classes, by considering the relaxed notion of nonuniform CPAC learning.