This looks like what? Challenges and Future Research Directions for Part-Prototype Models

TL;DR

This paper addresses the gap between the intrinsic interpretability of part-prototype models (PPMs) and their practical adoption. By surveying 45 PPM-related works from 2019–2024, it builds a four-category taxonomy of challenges (Prototypes, Methodology, Generalization, Safety) and articulates five concrete research directions to close the interpretability–performance gap. The authors highlight issues around prototype quantity/quality, theoretical grounding, evaluation standards, and real-world safety, proposing human-AI collaboration, theory-driven architectures, and standardized benchmarks as central remedies. The work aims to catalyze the development of intrinsically interpretable models capable of safe deployment in high-stakes domains.

Abstract

The growing interest in eXplainable Artificial Intelligence (XAI) has prompted research into models with built-in interpretability, the most prominent of which are part-prototype models. Part-Prototype Models (PPMs) make decisions by comparing an input image to a set of learned prototypes, providing human-understandable explanations in the form of ``this looks like that''. Despite their inherent interpretability, PPMS are not yet considered a valuable alternative to post-hoc models. In this survey, we investigate the reasons for this and provide directions for future research. We analyze papers from 2019 to 2024, and derive a taxonomy of the challenges that current PPMS face. Our analysis shows that the open challenges are quite diverse. The main concern is the quality and quantity of prototypes. Other concerns are the lack of generalization to a variety of tasks and contexts, and general methodological issues, including non-standardized evaluation. We provide ideas for future research in five broad directions: improving predictive performance, developing novel architectures grounded in theory, establishing frameworks for human-AI collaboration, aligning models with humans, and establishing metrics and benchmarks for evaluation. We hope that this survey will stimulate research and promote intrinsically interpretable models for application domains. Our list of surveyed papers is available at https://github.com/aix-group/ppm-survey.

Figures (4)

• Figure 1: Overview of the reasoning process of part-prototype models (top) and selected challenges (bottom). Part-prototype models first find occurrences of learned prototypes (different types of birds' beaks, breasts and heads) in an input image (prototype matching) by calculating the similarity of input patches and each prototype. The decision layer is a simple scoring sheet, integrating these similarities and learned weights to classes. There are multiple open challenges: It can be unclear why an image patch is matched to a certain prototype (a), the spatial alignment of the matching is not optimal (b), prototypes are learned, which have no meaning to humans (c) or should be irrelevant for the task (d). Additionally, the number of prototypes is too high (e) or redundant prototypes are learned (f). Further, there is a lack in clarity about what a prototype is in modalities other than images, e.g., for sensor data or text (g). Best viewed in color.
• Figure 2: Corpus overview: a) Number of papers per modality (Seq. - Sequences). b) Number of papers per venue and year. Survey papers are marked with a star pattern. Note: The seed paper on ProtoPNet Chen_2019_ThisLooksThat was the only paper published in 2019, and first subsequent work appeared in 2021 in our list of venues.
• Figure 3: Taxonomy of challenges that current (2024) part-prototype models face.
• Figure 4: Principle directions for future research. In addition to technical and theoretical research (top row), it is important to address human-centered issues (bottom row).