Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ABD: Default Exception Abduction in Finite First Order Worlds

Serafim Batzoglou

TL;DR

ABD is introduced, a benchmark for default-exception abduction over finite first-order worlds, and three observation regimes are formalized with exact SMT verification with exact SMT verification.

Abstract

We introduce ABD, a benchmark for default-exception abduction over finite first-order worlds. Given a background theory with an abnormality predicate and a set of relational structures, a model must output a first-order formula that defines exceptions, restoring satisfiability while keeping exceptions sparse. We formalize three observation regimes (closed-world, existential completion, universal completion) with exact SMT verification. Evaluating ten frontier LLMs on 600 instances, the best models achieve high validity but parsimony gaps remain, and holdout evaluation reveals distinct generalization failure modes across regimes.

ABD: Default Exception Abduction in Finite First Order Worlds

TL;DR

ABD is introduced, a benchmark for default-exception abduction over finite first-order worlds, and three observation regimes are formalized with exact SMT verification with exact SMT verification.

Abstract

We introduce ABD, a benchmark for default-exception abduction over finite first-order worlds. Given a background theory with an abnormality predicate and a set of relational structures, a model must output a first-order formula that defines exceptions, restoring satisfiability while keeping exceptions sparse. We formalize three observation regimes (closed-world, existential completion, universal completion) with exact SMT verification. Evaluating ten frontier LLMs on 600 instances, the best models achieve high validity but parsimony gaps remain, and holdout evaluation reveals distinct generalization failure modes across regimes.
Paper Structure (141 sections, 24 equations, 1 figure, 25 tables)

This paper contains 141 sections, 24 equations, 1 figure, 25 tables.

Table of Contents

  1. Introduction
  2. Three Observation Regimes.
  3. ABD-Full
  4. Evaluation Beyond Validity.
  5. Main Empirical Findings.
  6. Contributions.
  7. Related Work
  8. Abduction, Defaults, and Non-Monotonic Reasoning.
  9. Abductive Logic Programming and Diagnosis.
  10. Inductive Logic Programming and Relational Learning.
  11. Program Synthesis and Solver-Aided Search.
  12. Logic and Relational Reasoning Benchmarks.
  13. Mathematical Reasoning and Formal Proof Benchmarks.
  14. Neuro-Symbolic Learning and Large Language Models.
  15. Abductive Reasoning in Natural Language.
  16. ...and 126 more sections

Figures (1)

  • Figure D.1: CEGIS-like adversarial world addition.