Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Human-Modeling in Sequential Decision-Making: An Analysis through the Lens of Human-Aware AI

Silvia Tulli, Stylianos Loukas Vasileiou, Sarath Sreedharan

TL;DR

This paper clarifies what constitutes human-aware AI in sequential decision-making by identifying two core design features: explicit acknowledgment of human interaction (F1) and design that actively models human behavior (F2). It provides a framework using two central human models, $\mathcal{M}^H$ and $\mathcal{M}^R_h$, to analyze how humans participate as supervisors, teammates, or end-users across recent AI conference work. Through a reproducible literature review of 2020–2023 papers, the authors find a strong emphasis on modeling the human knowledge state ($\mathcal{M}_h$) but relatively less on modeling the human's view of the agent ($\mathcal{M}^R_h$) or on vocabulary alignment and social-science grounding, with few studies involving user experiments. The work highlights gaps in explicit human modeling, theory-grounded evaluation, and user studies, offering concrete guidance for future research to advance practical, human-centered sequential decision-making systems.

Abstract

"Human-aware" has become a popular keyword used to describe a particular class of AI systems that are designed to work and interact with humans. While there exists a surprising level of consistency among the works that use the label human-aware, the term itself mostly remains poorly understood. In this work, we retroactively try to provide an account of what constitutes a human-aware AI system. We see that human-aware AI is a design oriented paradigm, one that focuses on the need for modeling the humans it may interact with. Additionally, we see that this paradigm offers us intuitive dimensions to understand and categorize the kinds of interactions these systems might have with humans. We show the pedagogical value of these dimensions by using them as a tool to understand and review the current landscape of work related to human-AI systems that purport some form of human modeling. To fit the scope of a workshop paper, we specifically narrowed our review to papers that deal with sequential decision-making and were published in a major AI conference in the last three years. Our analysis helps identify the space of potential research problems that are currently being overlooked. We perform additional analysis on the degree to which these works make explicit reference to results from social science and whether they actually perform user-studies to validate their systems. We also provide an accounting of the various AI methods used by these works.

Human-Modeling in Sequential Decision-Making: An Analysis through the Lens of Human-Aware AI

TL;DR

This paper clarifies what constitutes human-aware AI in sequential decision-making by identifying two core design features: explicit acknowledgment of human interaction (F1) and design that actively models human behavior (F2). It provides a framework using two central human models, and , to analyze how humans participate as supervisors, teammates, or end-users across recent AI conference work. Through a reproducible literature review of 2020–2023 papers, the authors find a strong emphasis on modeling the human knowledge state () but relatively less on modeling the human's view of the agent () or on vocabulary alignment and social-science grounding, with few studies involving user experiments. The work highlights gaps in explicit human modeling, theory-grounded evaluation, and user studies, offering concrete guidance for future research to advance practical, human-centered sequential decision-making systems.

Abstract

"Human-aware" has become a popular keyword used to describe a particular class of AI systems that are designed to work and interact with humans. While there exists a surprising level of consistency among the works that use the label human-aware, the term itself mostly remains poorly understood. In this work, we retroactively try to provide an account of what constitutes a human-aware AI system. We see that human-aware AI is a design oriented paradigm, one that focuses on the need for modeling the humans it may interact with. Additionally, we see that this paradigm offers us intuitive dimensions to understand and categorize the kinds of interactions these systems might have with humans. We show the pedagogical value of these dimensions by using them as a tool to understand and review the current landscape of work related to human-AI systems that purport some form of human modeling. To fit the scope of a workshop paper, we specifically narrowed our review to papers that deal with sequential decision-making and were published in a major AI conference in the last three years. Our analysis helps identify the space of potential research problems that are currently being overlooked. We perform additional analysis on the degree to which these works make explicit reference to results from social science and whether they actually perform user-studies to validate their systems. We also provide an accounting of the various AI methods used by these works.
Paper Structure (12 sections, 1 figure, 1 table)

This paper contains 12 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Humans and Human-Aware AI System
  3. Methodology
  4. Human Assumptions
  5. Human Models
  6. Priori/Posteriori Inclusion of the Human
  7. Role of the Human
  8. Social Science Theories
  9. Evaluation Methods and Metrics
  10. AI Methods and Learning Paradigms
  11. Takeaways
  12. Conclusion and Future Directions

Figures (1)

  • Figure 1: A visualization of the three modes in which the human and AI systems could interact and the models that would come into play in such interactions (the most important one is bolded and put in a slightly larger font). In the context of teaming or collaboration, the three categories correspond to (A) Supervisor/Teacher (B) Teammate (C) End-User.