Table of Contents
Fetching ...

Predicate Renaming via Large Language Models

Elisabetta Gentili, Tony Ribeiro, Fabrizio Riguzzi, Katsumi Inoue

TL;DR

Predicate renaming for unnamed predicates in ILP is tackled with a pipeline that uses large language models to propose semantically meaningful names. The approach combines zero-shot prompts, a structured naming workflow, and LLMs as judges plus human validation across multiple case studies including head-only, head-and-body, body-only, reachability, and mutagenesis data. Results show that several state-of-the-art LLMs can consistently produce plausible names and that LLM-based judging aligns with human judgments in many cases. The work demonstrates the practicality of automated predicate naming to improve readability and reuse of logic programs, and points to domain-specific fine-tuning as future work.

Abstract

In this paper, we address the problem of giving names to predicates in logic rules using Large Language Models (LLMs). In the context of Inductive Logic Programming, various rule generation methods produce rules containing unnamed predicates, with Predicate Invention being a key example. This hinders the readability, interpretability, and reusability of the logic theory. Leveraging recent advancements in LLMs development, we explore their ability to process natural language and code to provide semantically meaningful suggestions for giving a name to unnamed predicates. The evaluation of our approach on some hand-crafted logic rules indicates that LLMs hold potential for this task.

Predicate Renaming via Large Language Models

TL;DR

Predicate renaming for unnamed predicates in ILP is tackled with a pipeline that uses large language models to propose semantically meaningful names. The approach combines zero-shot prompts, a structured naming workflow, and LLMs as judges plus human validation across multiple case studies including head-only, head-and-body, body-only, reachability, and mutagenesis data. Results show that several state-of-the-art LLMs can consistently produce plausible names and that LLM-based judging aligns with human judgments in many cases. The work demonstrates the practicality of automated predicate naming to improve readability and reuse of logic programs, and points to domain-specific fine-tuning as future work.

Abstract

In this paper, we address the problem of giving names to predicates in logic rules using Large Language Models (LLMs). In the context of Inductive Logic Programming, various rule generation methods produce rules containing unnamed predicates, with Predicate Invention being a key example. This hinders the readability, interpretability, and reusability of the logic theory. Leveraging recent advancements in LLMs development, we explore their ability to process natural language and code to provide semantically meaningful suggestions for giving a name to unnamed predicates. The evaluation of our approach on some hand-crafted logic rules indicates that LLMs hold potential for this task.

Paper Structure

This paper contains 31 sections, 1 equation, 6 figures, 12 tables.

Figures (6)

  • Figure 1: Diagram illustrating key components and interactions of the approach we propose.
  • Figure 2: Prompt 1 for asking for one suggestion for each unnamed predicate. This prompt was used for Case Study \ref{['cs:family']}. [rules] is replaced with the corresponding rules.
  • Figure 3: Prompt 2 used for asking the models to choose one among their own suggestions. This prompt was used for Case Study \ref{['cs:family']}. [rules] is replaced with the corresponding rules and the suggestions of the model being used replace [suggested names].
  • Figure 4: Prompt 3 used for asking the models acting as judges to score the suggestions for each predicate. This prompt was used for Case Study \ref{['cs:family']}. [rules] is replaced with the corresponding rules.
  • Figure 5: Result obtained with Command R+ on Case Study \ref{['cs:muta_example_rules']} with a less constrained version of Prompt 1.
  • ...and 1 more figures

Theorems & Definitions (3)

  • Example 1
  • Example 2
  • Example 3