Program Synthesis using Inductive Logic Programming for the Abstraction and Reasoning Corpus

TL;DR

ARC is a general AI benchmark that remains challenging for neural models. This work proposes a program synthesis approach using Inductive Logic Programming (ILP) built on a hand-crafted object-centric DSL as Background Knowledge to generate Output-grid transformations. The system learns Logic Programs that generate Output-grid objects and composes them into full solutions, leveraging divide-and-conquer and deductive search to manage the search space and enable generalization from few examples. Preliminary results on five ARC tasks illustrate the potential of symbolic, interpretable learning for generalizing solution strategies beyond what current neural methods achieve.

Abstract

The Abstraction and Reasoning Corpus (ARC) is a general artificial intelligence benchmark that is currently unsolvable by any Machine Learning method, including Large Language Models (LLMs). It demands strong generalization and reasoning capabilities which are known to be weaknesses of Neural Network based systems. In this work, we propose a Program Synthesis system that uses Inductive Logic Programming (ILP), a branch of Symbolic AI, to solve ARC. We have manually defined a simple Domain Specific Language (DSL) that corresponds to a small set of object-centric abstractions relevant to ARC. This is the Background Knowledge used by ILP to create Logic Programs that provide reasoning capabilities to our system. The full system is capable of generalize to unseen tasks, since ILP can create Logic Program(s) from few examples, in the case of ARC: pairs of Input-Output grids examples for each task. These Logic Programs are able to generate Objects present in the Output grid and the combination of these can form a complete program that transforms an Input grid into an Output grid. We randomly chose some tasks from ARC that dont require more than the small number of the Object primitives we implemented and show that given only these, our system can solve tasks that require each, such different reasoning.

Figures (12)

• Figure 1: Example tasks of the ARC Training dataset with the solutions shown. The goal is to produce the Test Output grid given the Test Input grid and the Train Input-Output grid pairs. We can see the logic behind each task is very different and the relations between objects are key to the solutions.
• Figure 2: Example of an Input grid with a Rectangle (or with several contiguous Points or Lines) and three possible Output grids, built from the Input, depending on which type of Object representation is used. The Output object can be described by the relation Copy applied to the Input Rectangle or to just one Input Point or Input Line that is part of the same Rectangle.
• Figure 3: Example of an Input-Output transformation when applied a LineFromPoint Logic Program that is able to generate Lines from Points.
• Figure 4: Example of an Input-Output transformation when applied a LineFromPoint Logic Program without having the Orientation defined.
• Figure 5: Example task that contains two Train examples with vertical lines and two Train examples with horizontal lines, in the Output grids. The Test example only requires vertical Lines in the same way as the first two Train examples.