Learning Conditional Averages

TL;DR

This work introduces and resolves the problem of learning conditional averages over neighborhoods within the PAC framework. It catalogs learnability via two joint combinatorial parameters ${\alpha}_1(G,{\mathcal C})$ and ${\alpha}_2(G,{\mathcal C})$, proving learnability iff their sum is finite and providing tight (up to log factors) sample complexity bounds. The main algorithm combines empirical neighborhood averaging for adjacent test points with the One-Inclusion Graph method on isolated points, achieving both in-expectation and high-probability guarantees. The results illuminate when conditional averages can be learned, reveal limits of VC-based reasoning in this setting, and connect to applications in explainability, fairness, and recommendation systems.

Abstract

We introduce the problem of learning conditional averages in the PAC framework. The learner receives a sample labeled by an unknown target concept from a known concept class, as in standard PAC learning. However, instead of learning the target concept itself, the goal is to predict, for each instance, the average label over its neighborhood -- an arbitrary subset of points that contains the instance. In the degenerate case where all neighborhoods are singletons, the problem reduces exactly to classic PAC learning. More generally, it extends PAC learning to a setting that captures learning tasks arising in several domains, including explainability, fairness, and recommendation systems. Our main contribution is a complete characterization of when conditional averages are learnable, together with sample complexity bounds that are tight up to logarithmic factors. The characterization hinges on the joint finiteness of two novel combinatorial parameters, which depend on both the concept class and the neighborhood system, and are closely related to the independence number of the associated neighborhood graph.

Theorems & Definitions (20)

• Definition 1: $(G,{\mathcal{C}})$-learner
• Definition 2: Parameters $\alpha_1$ and $\alpha_2$
• Theorem 3
• Corollary 4: Full class
• Corollary 5: Singleton class
• Theorem 6
• Lemma 6
• Lemma 7: Light-neighborhood nodes are light
• Proposition 8
• Theorem 9