EXPLAIN, AGREE, LEARN: Scaling Learning for Neural Probabilistic Logic
Victor Verreet, Lennert De Smet, Luc De Raedt, Emanuele Sansone
TL;DR
This work tackles the challenge of scalable learning in neural probabilistic logic (NeSy) by introducing a sampling-based surrogate objective $L = \sum_i \mathrm{KL}(Q_i \| P_i)$ that bounds the true data likelihood and avoids expensive exact inference. The EXPLAIN, AGREE, LEARN (EXAL) framework constructs this surrogate by sampling explanations, reweighting them with the neural posterior, and performing standard gradient updates on the neural component. The authors prove optimality and bounding relationships for the surrogate, show that diversity among explanations tightens the bound, and reformulate sampling as a Markov decision process optimized via GFlowNets. Empirically, EXAL demonstrates faster convergence and competitive accuracy on MNIST addition and Warcraft pathfinding compared with state-of-the-art NeSy methods, while providing theoretical guarantees on approximation error. This approach enables scalable NeSy learning with probabilistic logic by leveraging diverse, sampled explanations to guide neural updates.
Abstract
Neural probabilistic logic systems follow the neuro-symbolic (NeSy) paradigm by combining the perceptive and learning capabilities of neural networks with the robustness of probabilistic logic. Learning corresponds to likelihood optimization of the neural networks. However, to obtain the likelihood exactly, expensive probabilistic logic inference is required. To scale learning to more complex systems, we therefore propose to instead optimize a sampling based objective. We prove that the objective has a bounded error with respect to the likelihood, which vanishes when increasing the sample count. Furthermore, the error vanishes faster by exploiting a new concept of sample diversity. We then develop the EXPLAIN, AGREE, LEARN (EXAL) method that uses this objective. EXPLAIN samples explanations for the data. AGREE reweighs each explanation in concordance with the neural component. LEARN uses the reweighed explanations as a signal for learning. In contrast to previous NeSy methods, EXAL can scale to larger problem sizes while retaining theoretical guarantees on the error. Experimentally, our theoretical claims are verified and EXAL outperforms recent NeSy methods when scaling up the MNIST addition and Warcraft pathfinding problems.