Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompt2DeModel: Declarative Neuro-Symbolic Modeling with Natural Language

Hossein Rajaby Faghihi, Aliakbar Nafar, Andrzej Uszok, Hamid Karimian, Parisa Kordjamshidi

TL;DR

The paper tackles enabling domain experts to declaratively inject structured knowledge into neuro-symbolic models via natural language. It introduces an interactive NL-to-DomiKnowS pipeline that uses large language models to generate declarative programs, maps NL descriptions to First-Order Logic through intermediary steps, and iteratively refines outputs with a symbolic parser feedback loop. The approach yields a concept graph and logical constraints that integrate with PyTorch-based neural components, demonstrated across NLP, vision, and CSP tasks. Results indicate improved automatic and human-evaluated accuracy and substantial reductions in user effort, supporting practical customization of complex models by non-ML experts.

Abstract

This paper presents a conversational pipeline for crafting domain knowledge for complex neuro-symbolic models through natural language prompts. It leverages large language models to generate declarative programs in the DomiKnowS framework. The programs in this framework express concepts and their relationships as a graph in addition to logical constraints between them. The graph, later, can be connected to trainable neural models according to those specifications. Our proposed pipeline utilizes techniques like dynamic in-context demonstration retrieval, model refinement based on feedback from a symbolic parser, visualization, and user interaction to generate the tasks' structure and formal knowledge representation. This approach empowers domain experts, even those not well-versed in ML/AI, to formally declare their knowledge to be incorporated in customized neural models in the DomiKnowS framework.

Prompt2DeModel: Declarative Neuro-Symbolic Modeling with Natural Language

TL;DR

The paper tackles enabling domain experts to declaratively inject structured knowledge into neuro-symbolic models via natural language. It introduces an interactive NL-to-DomiKnowS pipeline that uses large language models to generate declarative programs, maps NL descriptions to First-Order Logic through intermediary steps, and iteratively refines outputs with a symbolic parser feedback loop. The approach yields a concept graph and logical constraints that integrate with PyTorch-based neural components, demonstrated across NLP, vision, and CSP tasks. Results indicate improved automatic and human-evaluated accuracy and substantial reductions in user effort, supporting practical customization of complex models by non-ML experts.

Abstract

This paper presents a conversational pipeline for crafting domain knowledge for complex neuro-symbolic models through natural language prompts. It leverages large language models to generate declarative programs in the DomiKnowS framework. The programs in this framework express concepts and their relationships as a graph in addition to logical constraints between them. The graph, later, can be connected to trainable neural models according to those specifications. Our proposed pipeline utilizes techniques like dynamic in-context demonstration retrieval, model refinement based on feedback from a symbolic parser, visualization, and user interaction to generate the tasks' structure and formal knowledge representation. This approach empowers domain experts, even those not well-versed in ML/AI, to formally declare their knowledge to be incorporated in customized neural models in the DomiKnowS framework.
Paper Structure (12 sections, 3 figures, 2 tables)

This paper contains 12 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Research
  3. DomiKnowS Interface
  4. Graph Structure
  5. Logical Constraints
  6. Natural Language to Knowledge Declaration
  7. Evaluation and Analysis
  8. Automatic Evaluation
  9. Human Evaluation
  10. Interface V.S. Coding
  11. Interactive Setting
  12. Conclusion

Figures (3)

  • Figure 1: Parts of the concept graph generated for the Natural Language Inference task.
  • Figure 2: Overview of the pipeline: The purple, green, and red parts represent direct user inputs, dynamic in-context examples, and execution/parser feedback, respectively.
  • Figure 3: The visualized graph for the NLI task.