Symbolic Snapshot Ensembles
Mingyue Liu, Andrew Cropper
TL;DR
Symbolic Snapshot Ensembles address the ILP limitation of learning a single hypothesis by harvesting intermediate hypotheses from one anytime ILP run. These hypotheses are pooled and weighted using a minimum description length ($MDL$) framework to balance fit and complexity, producing a weighted ensemble that predicts with a 0.5 threshold. Across 111 tasks covering game playing and visual reasoning, the approach yields about a $4\%$ accuracy gain with under $1\%$ additional cost, and often matches or surpasses traditional bagging while being far more computationally efficient. The work demonstrates that diverse, structurally distinct rule sets generated during a single search can be effectively aggregated to boost generalisation in symbolic learning.
Abstract
Inductive logic programming (ILP) is a form of logical machine learning. Most ILP algorithms learn a single hypothesis from a single training run. Ensemble methods train an ILP algorithm multiple times to learn multiple hypotheses. In this paper, we train an ILP algorithm only once and save intermediate hypotheses. We then combine the hypotheses using a minimum description length weighting scheme. Our experiments on multiple benchmarks, including game playing and visual reasoning, show that our approach improves predictive accuracy by 4% with less than 1% computational overhead.