Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Constraint Compliant Goal Formulation and Planning

Steven J. Jones, Robert E. Wray

TL;DR

This paper addresses constraint-compliant goal formulation and planning for autonomous agents operating under ethical norms in uncertain environments. It adopts a constraint-compliance pipeline built on Soar and uses an A* planner with two ethical framings, utilitarian and deontological, to study behavior in a grid-based maritime warehouse domain. Metacognitive mechanisms are introduced to resolve conflicts when norms and tasks clash, enabling relaxation of norms or re-planning. The findings show that ethical framing qualitatively changes path choice and planning costs, and that metacognition improves flexibility, offering guidance for designing constraint-aware autonomous agents and future work toward richer ethical theories.

Abstract

One part of complying with norms, rules, and preferences is incorporating constraints (such as knowledge of ethics) into one's goal formulation and planning processing. We explore in a simple domain how the encoding of knowledge in different ethical frameworks influences an agent's goal formulation and planning processing and demonstrate ability of an agent to satisfy and satisfice when its collection of relevant constraints includes a mix of "hard" and "soft" constraints of various types. How the agent attempts to comply with ethical constraints depends on the ethical framing and we investigate tradeoffs between deontological framing and utilitarian framing for complying with an ethical norm. Representative scenarios highlight how performing the same task with different framings of the same norm leads to different behaviors. Our explorations suggest an important role for metacognitive judgments in resolving ethical conflicts during goal formulation and planning.

Toward Constraint Compliant Goal Formulation and Planning

TL;DR

This paper addresses constraint-compliant goal formulation and planning for autonomous agents operating under ethical norms in uncertain environments. It adopts a constraint-compliance pipeline built on Soar and uses an A* planner with two ethical framings, utilitarian and deontological, to study behavior in a grid-based maritime warehouse domain. Metacognitive mechanisms are introduced to resolve conflicts when norms and tasks clash, enabling relaxation of norms or re-planning. The findings show that ethical framing qualitatively changes path choice and planning costs, and that metacognition improves flexibility, offering guidance for designing constraint-aware autonomous agents and future work toward richer ethical theories.

Abstract

One part of complying with norms, rules, and preferences is incorporating constraints (such as knowledge of ethics) into one's goal formulation and planning processing. We explore in a simple domain how the encoding of knowledge in different ethical frameworks influences an agent's goal formulation and planning processing and demonstrate ability of an agent to satisfy and satisfice when its collection of relevant constraints includes a mix of "hard" and "soft" constraints of various types. How the agent attempts to comply with ethical constraints depends on the ethical framing and we investigate tradeoffs between deontological framing and utilitarian framing for complying with an ethical norm. Representative scenarios highlight how performing the same task with different framings of the same norm leads to different behaviors. Our explorations suggest an important role for metacognitive judgments in resolving ethical conflicts during goal formulation and planning.
Paper Structure (14 sections, 5 figures, 2 tables, 1 algorithm)

This paper contains 14 sections, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Experimental Domain and Task
  4. Planning with Ethical Constraints
  5. Baseline Planning Algorithm: A*
  6. Adapting Planning to Accommodate Norms: Alternative Ethical Framings
  7. Utilitarianism
  8. Deontology
  9. Resolving Conflicts via Metacognition
  10. Knowledge for Conflict with Deontological Norm
  11. Knowledge for Conflict with Utilitarian Norm
  12. Results and Discussion
  13. Related Work
  14. Conclusion

Figures (5)

  • Figure 1: A computational pipeline for constraint compliance wray_computational-level_2023.
  • Figure 2: The planned path depends on movement cost and the cost of entering a cell with a person.
  • Figure 3: The agent's patrol paths, when using a utilitarian framing of the personal space norm in both the consensus and dilemma scenarios, are depicted with blue arrows. Dashed arrows indicate where the route the agent plans and executes depends on the encoded cost associated with personal space violation.
  • Figure 4: The agent's patrol paths are depicted with blue arrows when using a deontological framing of the personal space norm in both the consensus and dilemma scenarios.
  • Figure 5: This agent uses a utilitarian framing of the ethical constraint and includes sensitivity to constraints about crates and barrels. The agent plans the route on the left when using a personal space cost of 2 and the route on the right when using a personal space cost of 6.