Computable learning of natural hypothesis classes
Matthew Harrison-Trainor, Syed Akbari
TL;DR
The paper investigates whether natural PAC-learnable hypothesis classes must admit computable (CPAC) learners when analyzed via the on-a-cone framework from computability theory. It develops positive results for c.e.-represented and closed hypothesis classes, showing that PAC-learnability on a cone implies properly SCPAC-learnability on a cone, and extends these insights under determinacy assumptions to obtain improper SCPAC on a cone. It also delineates the limits of this program by constructing natural PAC-learnable but not CPAC examples outside the stated conditions, and by examining more delicate representations (e.g., Sigma^0_2) where CPAC failures persist. Overall, the work clarifies when computability constraints force CPAC learning for natural classes and highlights the role of topological and set-theoretic assumptions in bridging statistical and computable learning theories.
Abstract
This paper is about the recent notion of computably probably approximately correct learning, which lies between the statistical learning theory where there is no computational requirement on the learner and efficient PAC where the learner must be polynomially bounded. Examples have recently been given of hypothesis classes which are PAC learnable but not computably PAC learnable, but these hypothesis classes are unnatural or non-canonical in the sense that they depend on a numbering of proofs, formulas, or programs. We use the on-a-cone machinery from computability theory to prove that, under mild assumptions such as that the hypothesis class can be computably listable, any natural hypothesis class which is learnable must be computably learnable. Thus the counterexamples given previously are necessarily unnatural.