Understanding Domain-Size Generalization in Markov Logic Networks

TL;DR

This work tackles domain-size generalization for Markov Logic Networks by analyzing how learned parameters may fail to generalize across relational structures of different sizes, owing to non-projectivity. It develops a variance-driven, KL-divergence based framework that bounds the difference between marginals on small and large domains, showing that minimizing parameter variance (via regularization or Domain-Size Aware MLNs) aligns with better cross-domain generalization. The authors establish theoretical connections that apply to Exponential Random Graph Models and other SRL models with template-based parameter sharing, and they validate the approach empirically using four datasets with variance-reduction methods (L1, L2, DA-MLNs) to demonstrate improved target-domain likelihoods. The results provide principled guidance for learning MLNs that generalize across domain sizes and offer practical implications for SRL applications in relational data settings.

Abstract

We study the generalization behavior of Markov Logic Networks (MLNs) across relational structures of different sizes. Multiple works have noticed that MLNs learned on a given domain generalize poorly across domains of different sizes. This behavior emerges from a lack of internal consistency within an MLN when used across different domain sizes. In this paper, we quantify this inconsistency and bound it in terms of the variance of the MLN parameters. The parameter variance also bounds the KL divergence between an MLN's marginal distributions taken from different domain sizes. We use these bounds to show that maximizing the data log-likelihood while simultaneously minimizing the parameter variance corresponds to two natural notions of generalization across domain sizes. Our theoretical results apply to Exponential Random Graphs and other Markov network based relational models. Finally, we observe that solutions known to decrease the variance of the MLN parameters, like regularization and Domain-Size Aware MLNs, increase the internal consistency of the MLNs. We empirically verify our results on four different datasets, with different methods to control parameter variance, showing that controlling parameter variance leads to better generalization.