Conditionality principle under unconstrained randomness
Vladimir Vovk
TL;DR
The paper examines the conflict between Fisher's conditionality principle and predictive goals in statistical learning, showing that conditioning on observed inputs under unrestricted randomness can render prediction impossible. It analyzes conformal prediction as a method to obtain valid predictive sets without strict conditioning and surveys various notions of conditional validity, highlighting both impossibility results and practical weak-conditional approaches. The work argues for relaxing conditionality in machine learning to enable reliable prediction with finite-sample guarantees, using conformal methods when unconditional guarantees are preferred. It clarifies how different forms of conditional reasoning impact predictive performance and guidance for practical methodology.
Abstract
A very simple example demonstrates that Fisher's application of the conditionality principle to regression ("fixed-$x$ regression"), endorsed by Sprott and many other followers, makes prediction impossible in the context of statistical learning theory. On the other hand, relaxing the requirement of conditionality makes it possible via, e.g., conformal prediction.