Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Should agentic conversational AI change how we think about ethics? Characterising an interactional ethics centred on respect

Lize Alberts, Geoff Keeling, Amanda McCroskery

TL;DR

This work reframes AI ethics for agentic conversational systems by treating them as social actors whose behavior in ongoing interactions, not just output quality, shapes ethical outcomes. It introduces an interactional ethics centered on respect, drawing from philosophy, psychology, and bioethics to identify three families of social-interactional harms: direct harms, influence harms, and cumulative harms. Respect is operationalized through three duties—affirming autonomy, competence, and self-worth—supported by design guidance for embedding these duties in LLM agents and memory management, with consideration of GUI-based interactions to reduce manipulation risk. By connecting to HCI approaches (value-sensitive design, user-centered design) and ethics of care, the paper offers a normative framework that guides the development of agentic AI toward treating users with appropriate regard, ultimately enhancing wellbeing in situated interactions. The practical significance lies in anticipating interaction-level risks in agentic systems and providing actionable design principles to align AI behavior with human psychological needs and social norms.

Abstract

With the growing popularity of conversational agents based on large language models (LLMs), we need to ensure their behaviour is ethical and appropriate. Work in this area largely centres around the 'HHH' criteria: making outputs more helpful and honest, and avoiding harmful (biased, toxic, or inaccurate) statements. Whilst this semantic focus is useful when viewing LLM agents as mere mediums or output-generating systems, it fails to account for pragmatic factors that can make the same speech act seem more or less tactless or inconsiderate in different social situations. With the push towards agentic AI, wherein systems become increasingly proactive in chasing goals and performing actions in the world, considering the pragmatics of interaction becomes essential. We propose an interactional approach to ethics that is centred on relational and situational factors. We explore what it means for a system, as a social actor, to treat an individual respectfully in a (series of) interaction(s). Our work anticipates a set of largely unexplored risks at the level of situated social interaction, and offers practical suggestions to help agentic LLM technologies treat people well.

Should agentic conversational AI change how we think about ethics? Characterising an interactional ethics centred on respect

TL;DR

This work reframes AI ethics for agentic conversational systems by treating them as social actors whose behavior in ongoing interactions, not just output quality, shapes ethical outcomes. It introduces an interactional ethics centered on respect, drawing from philosophy, psychology, and bioethics to identify three families of social-interactional harms: direct harms, influence harms, and cumulative harms. Respect is operationalized through three duties—affirming autonomy, competence, and self-worth—supported by design guidance for embedding these duties in LLM agents and memory management, with consideration of GUI-based interactions to reduce manipulation risk. By connecting to HCI approaches (value-sensitive design, user-centered design) and ethics of care, the paper offers a normative framework that guides the development of agentic AI toward treating users with appropriate regard, ultimately enhancing wellbeing in situated interactions. The practical significance lies in anticipating interaction-level risks in agentic systems and providing actionable design principles to align AI behavior with human psychological needs and social norms.

Abstract

With the growing popularity of conversational agents based on large language models (LLMs), we need to ensure their behaviour is ethical and appropriate. Work in this area largely centres around the 'HHH' criteria: making outputs more helpful and honest, and avoiding harmful (biased, toxic, or inaccurate) statements. Whilst this semantic focus is useful when viewing LLM agents as mere mediums or output-generating systems, it fails to account for pragmatic factors that can make the same speech act seem more or less tactless or inconsiderate in different social situations. With the push towards agentic AI, wherein systems become increasingly proactive in chasing goals and performing actions in the world, considering the pragmatics of interaction becomes essential. We propose an interactional approach to ethics that is centred on relational and situational factors. We explore what it means for a system, as a social actor, to treat an individual respectfully in a (series of) interaction(s). Our work anticipates a set of largely unexplored risks at the level of situated social interaction, and offers practical suggestions to help agentic LLM technologies treat people well.
Paper Structure (21 sections, 1 figure, 2 tables)

This paper contains 21 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Evaluating a system as a social actor
  3. Social-interactional harms
  4. Interactions that directly harm the user
  5. Interactions that harmfully influence user behaviour
  6. Interactions that collectively harm the user
  7. Respect as an evaluative lens for interactions
  8. What does respectful treatment mean?
  9. The moralised sense of recognition respect
  10. Respect and the ethics of care
  11. Respect and basic psychological needs theory
  12. Respect and human-computer interaction
  13. Critical integration: respect as interactional duties
  14. Affirming a person's sense of autonomy
  15. Affirming a person's sense of competence
  16. ...and 6 more sections

Figures (1)

  • Figure 1: Levels of analysing AI agents