Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Supporting AI-Augmented Meta-Decision Making with InDecision

Chance Castañeda, Jessica Mindel, Will Page, Hayden Stec, Manqing Yu, Kenneth Holstein

TL;DR

The paper addresses the challenge of guiding high-stakes decisions by explicitly developing decision criteria, highlighting that the process is under-supported and susceptible to criteria drift. It proposes design goals for AI tools to augment meta-decision making—treating humans as judges and AI as provocateur, providing concrete option prompts, varying the decision space, and preserving reflection, while mitigating conformity pressures—and illustrates these ideas with InDecision, a mixed-initiative tool that iteratively prototypes criteria by stress-testing them against simulated options. The authors report initial findings from pilot co-design with 11 participants, yielding insights on fluid workflow transitions, expressive criterion representations, expanded option spaces, and perceptions of AI versus human agency. The work suggests a viable path toward AI-assisted meta-decision making in domains like admissions, hiring, and investment, while outlining numerous open questions in design, collaboration, and safeguarding human judgment.

Abstract

From school admissions to hiring and investment decisions, the first step behind many high-stakes decision-making processes is "deciding how to decide." Formulating effective criteria to guide decision-making requires an iterative process of exploration, reflection, and discovery. Yet, this process remains under-supported in practice. In this short paper, we outline an opportunity space for AI-driven tools that augment human meta-decision making. We draw upon prior literature to propose a set of design goals for future AI tools aimed at supporting human meta-decision making. We then illustrate these ideas through InDecision, a mixed-initiative tool designed to support the iterative development of decision criteria. Based on initial findings from designing and piloting InDecision with users, we discuss future directions for AI-augmented meta-decision making.

Supporting AI-Augmented Meta-Decision Making with InDecision

TL;DR

The paper addresses the challenge of guiding high-stakes decisions by explicitly developing decision criteria, highlighting that the process is under-supported and susceptible to criteria drift. It proposes design goals for AI tools to augment meta-decision making—treating humans as judges and AI as provocateur, providing concrete option prompts, varying the decision space, and preserving reflection, while mitigating conformity pressures—and illustrates these ideas with InDecision, a mixed-initiative tool that iteratively prototypes criteria by stress-testing them against simulated options. The authors report initial findings from pilot co-design with 11 participants, yielding insights on fluid workflow transitions, expressive criterion representations, expanded option spaces, and perceptions of AI versus human agency. The work suggests a viable path toward AI-assisted meta-decision making in domains like admissions, hiring, and investment, while outlining numerous open questions in design, collaboration, and safeguarding human judgment.

Abstract

From school admissions to hiring and investment decisions, the first step behind many high-stakes decision-making processes is "deciding how to decide." Formulating effective criteria to guide decision-making requires an iterative process of exploration, reflection, and discovery. Yet, this process remains under-supported in practice. In this short paper, we outline an opportunity space for AI-driven tools that augment human meta-decision making. We draw upon prior literature to propose a set of design goals for future AI tools aimed at supporting human meta-decision making. We then illustrate these ideas through InDecision, a mixed-initiative tool designed to support the iterative development of decision criteria. Based on initial findings from designing and piloting InDecision with users, we discuss future directions for AI-augmented meta-decision making.

Paper Structure

This paper contains 10 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Toward AI-Augmented Meta-Decision Making
  3. InDecision: Helping People Develop Decision Criteria through Iterative Prototyping
  4. Design Exploration and Initial Findings
  5. A need for fluid transitions between iteration on options and iteration on criteria.
  6. Representing relationships across criteria.
  7. Expanding the space of options that decision-makers are able to consider.
  8. Is this for me, or the LLM?
  9. Conclusion and Open Questions
  10. Demo

Figures (1)

  • Figure 1: InDecision's iterative loop. The initial elicitation (1) allows the user to provide open-text descriptions of their decision and relevant considerations for options and criteria. The user is presented with a list of eight options (2). The user can keep, add to, or remove these options. To promote reflection on what is most important to them, the user may continue only after narrowing down to three. Criteria refinement is composed of two stages: (3a) prioritization, where a user can add, remove, and sort criteria in tiers of priority; and (3b) redefinition, where the user selects between a range of possible meanings associated with each criterion. These steps are repeated in an iterative loop (4).