A Measure Based Generalizable Approach to Understandability

TL;DR

This paper addresses the challenge that current agents struggle with understandability and steerability due to reliance on data-driven priors. It introduces a generalizable, domain-agnostic framework grounded in cognitive psychology to measure understandability across artifacts, organized into six dimensions: perceptual quality, memory cost, pattern decodability, cohesion, logical consistency, and semantic fit. By mapping existing domain-specific measures to these dimensions, the work provides a cohesive foundation for designing and evaluating more communicative and controllable agents. The framework aims to enable explicit optimization of understandability in real-world tasks and suggests future work on validation, representation, and extension to additional human priors and sensibilities, ultimately enhancing open-world planning and code maintainability.

Abstract

Successful agent-human partnerships require that any agent generated information is understandable to the human, and that the human can easily steer the agent towards a goal. Such effective communication requires the agent to develop a finer-level notion of what is understandable to the human. State-of-the-art agents, including LLMs, lack this detailed notion of understandability because they only capture average human sensibilities from the training data, and therefore afford limited steerability (e.g., requiring non-trivial prompt engineering). In this paper, instead of only relying on data, we argue for developing generalizable, domain-agnostic measures of understandability that can be used as directives for these agents. Existing research on understandability measures is fragmented, we survey various such efforts across domains, and lay a cognitive-science-rooted groundwork for more coherent and domain-agnostic research investigations in future.

Figures (1)

• Figure 1: Emerging landscape across models of understanding process