Conformal Prediction in Hierarchical Classification with Constrained Representation Complexity
Thomas Mortier, Alireza Javanmardi, Yusuf Sale, Eyke Hüllermeier, Willem Waegeman
TL;DR
This work addresses producing prediction sets with nominal coverage in hierarchical classification while maintaining interpretability. It extends split conformal prediction by introducing CRSVP with $R_{\mathcal{T}}(\hat{Y})=1$ and CRSVP-$r$ with $R_{\mathcal{T}}(\hat{Y})\le r$, each accompanied by calibration and inference procedures that provide distribution-free finite-sample guarantees. Empirical results across diverse benchmarks show that allowing representation complexity yields more efficient, semantically meaningful predictions, with randomized prediction sets ensuring exact coverage. The findings suggest that representation complexity can act as a regularizer in challenging probability landscapes and point to future work extending the framework to more complex structures like directed acyclic graphs.
Abstract
Conformal prediction has emerged as a widely used framework for constructing valid prediction sets in classification and regression tasks. In this work, we extend the split conformal prediction framework to hierarchical classification, where prediction sets are commonly restricted to internal nodes of a predefined hierarchy, and propose two computationally efficient inference algorithms. The first algorithm returns internal nodes as prediction sets, while the second one relaxes this restriction. Using the notion of representation complexity, the latter yields smaller set sizes at the cost of a more general and combinatorial inference problem. Empirical evaluations on several benchmark datasets demonstrate the effectiveness of the proposed algorithms in achieving nominal coverage.