Learning Symbolic Rules for Reasoning in Quasi-Natural Language
Kaiyu Yang, Jia Deng
TL;DR
The paper tackles the challenge of achieving interpretable, symbolic reasoning over natural language by introducing MetaQNL, a quasi-natural language framework for rule-based reasoning, and MetaInduce, a MAX-SAT–based method to induce MetaQNL rules from data. It demonstrates that compact rule sets can achieve strong reasoning performance across synthetic and real-world tasks while producing verifiable proofs. A key contribution is the combination of explicit symbolic rules with data-driven induction, enabling both interpretability and data efficiency. The work also explores soft matching to handle noisy or ambiguous inputs and discusses limitations and future integration with deep learning approaches for scalability and robustness.
Abstract
Symbolic reasoning, rule-based symbol manipulation, is a hallmark of human intelligence. However, rule-based systems have had limited success competing with learning-based systems outside formalized domains such as automated theorem proving. We hypothesize that this is due to the manual construction of rules in past attempts. In this work, we ask how we can build a rule-based system that can reason with natural language input but without the manual construction of rules. We propose MetaQNL, a "Quasi-Natural" language that can express both formal logic and natural language sentences, and MetaInduce, a learning algorithm that induces MetaQNL rules from training data consisting of questions and answers, with or without intermediate reasoning steps. Our approach achieves state-of-the-art accuracy on multiple reasoning benchmarks; it learns compact models with much less data and produces not only answers but also checkable proofs. Further, experiments on a real-world morphological analysis benchmark show that it is possible for our method to handle noise and ambiguity. Code will be released at https://github.com/princeton-vl/MetaQNL.