AI-Resilient Interfaces

TL;DR

This work defines AI-resilient interfaces as those that enable users to notice and judge AI choices that are objectively wrong, contextually inappropriate, or personally undesirable. It analyzes the cognitive and perceptual challenges in human-AI interaction, highlighting issues like inattentional blindness and the gulf of evaluation, and motivates resilient designs through motivating examples in AI-assisted search and document exploration. The paper presents GP-TSM as a concrete demonstration of resilience, showing how grammar-preserving text saliency modulation preserves context while exposing AI-driven saliency decisions, and discusses broader design principles and audits across AI tasks. The findings argue that increasing AI-resilience can improve safety, usability, and utility in open-ended AI-assisted tasks, albeit with trade-offs in cognitive load, suggesting a path toward task- and context-aware design guidelines.

Abstract

AI is powerful, but it can make choices that result in objective errors, contextually inappropriate outputs, and disliked options. We need AI-resilient interfaces that help people be resilient to the AI choices that are not right, or not right for them. To support this goal, interfaces need to help users notice and have the context to appropriately judge those AI choices. Existing human-AI interaction guidelines recommend efficient user dismissal, modification, or otherwise efficient recovery from AI choices that a user does not like. However, in order to recover from AI choices, the user must notice them first. This can be difficult. For example, when generating summaries of long documents, a system's exclusion of a detail that is critically important to the user is hard for the user to notice. That detail can be hiding in a wall of text in the original document, and the existence of a summary may tempt the user not to read the original document as carefully. Once noticed, judging AI choices well can also be challenging. The interface may provide very little information that contextualizes the choices, and the user may fall back on assumptions when deciding whether to dismiss, modify, or otherwise recover from an AI choice. Building on prior work, this paper defines key aspects of AI-resilient interfaces, illustrated with examples. Designing interfaces for increased AI-resilience of users will improve AI safety, usability, and utility. This is especially critical where AI-powered systems are used for context- and preference-dominated open-ended AI-assisted tasks, like ideating, summarizing, searching, sensemaking, and the reading and writing of text or code.

Figures (5)

• Figure 1: Generative AI's output in response to the query fraternal twins probability with maternal age.
• Figure 2: Another automated answer, which is a selected quote from the referenced page
• Figure 3: The actual referenced page babycenter. The blue and pink boxes highlight two distinct, critical pieces of information present on the original page that were automatically omitted from the extracted or generated answers on the original Google search page, misleading the searcher.
• Figure 4: An example of answers to the query folate daily for pregnancy as AI-extracted quotes initially provided with only the page title and organization name as context. The significant discrepancy in answers may be sufficient information scent for the user to take the necessary follow-up actions to be resilient to any of these AI choices that are not right for them.
• Figure 5: GP-TSM gptsm output, with multiple levels of text opacity revealing levels of AI-predicted semantic criticality, while keeping all original text (context) legible.