How does My Model Fail? Automatic Identification and Interpretation of Physical Plausibility Failure Modes with Matryoshka Transcoders

TL;DR

This work tackles the challenge of physical plausibility failures in generative models, which often evade standard evaluation. It introduces Matryoshka Transcoders, a hierarchical sparse-feature framework that learns from a physical plausibility classifier trained on human-annotated data and uses large multimodal models for interpretation. The approach yields automated, fine-grained, interpretable features of physical violations and establishes an interpretation-based benchmark across eight state-of-the-art generative models. By providing targeted insights into failure modes, the method supports more reliable and physically plausible image generation in high-stakes applications.

Abstract

Although recent generative models are remarkably capable of producing instruction-following and realistic outputs, they remain prone to notable physical plausibility failures. Though critical in applications, these physical plausibility errors often escape detection by existing evaluation methods. Furthermore, no framework exists for automatically identifying and interpreting specific physical error patterns in natural language, preventing targeted model improvements. We introduce Matryoshka Transcoders, a novel framework for the automatic discovery and interpretation of physical plausibility features in generative models. Our approach extends the Matryoshka representation learning paradigm to transcoder architectures, enabling hierarchical sparse feature learning at multiple granularity levels. By training on intermediate representations from a physical plausibility classifier and leveraging large multimodal models for interpretation, our method identifies diverse physics-related failure modes without manual feature engineering, achieving superior feature relevance and feature accuracy compared to existing approaches. We utilize the discovered visual patterns to establish a benchmark for evaluating physical plausibility in generative models. Our analysis of eight state-of-the-art generative models provides valuable insights into how these models fail to follow physical constraints, paving the way for further model improvements.

Figures (7)

• Figure 1: Matryoshka Transcoders for Automatic Physical Plausibility Analysis in Generative Models. We pass images from generative models to a physical plausibility classifier, whose representations are analyzed by Matryoshka Transcoders to extract hierarchical sparse features related to physical plausibility. These features encompasses anatomical impossibilities, spatial inconsistencies, gravity violations, and other physical implausibilities and are interpreted by LMMs automatically.
• Figure 2: Complete pipeline for our proposed method, Matryoshka Transcoders. Our approach combines four stages: (a) human annotation of correct/error images, (b) training a physical plausibility classifier that possess relevant information, (c) training Matryoshka transcoders to discover sparse, interpretable features at nested granularities from classifier activations, and (d) using large multimodal models to interpret discovered features by identifying visual commonalities and error explanations from maximally activating images.
• Figure 3: Qualitative examples of physical plausibility features discovered by Matryoshka Transcoders. Each column shows a representative feature with its top-activating images, automatically generated theme, error explanation, and categorized error type.
• Figure 4: Feature relevance distribution across methods. Percentage distribution of features by their opic-relevance score (wrong ratio) across four sparse dictionary learning methods. Features above the relevance threshold of 0.95 are considered as task-relevant features.
• Figure 5: Effect of Latent Dimension Size on Feature Discovery. Ablation study evaluating five dictionary sizes (1024-16384 latents). Smaller dictionaries achieve higher relevance scores (17.2% vs 11.6%) but discover fewer total relevant features (170 vs 1,896), revealing a quality-quantity tradeoff. Average wrong ratio remains stable ( 67-70%), indicating consistent feature selectivity across configurations. Nearly all features remain active (94-98%), showing efficient dictionary utilization regardless of size.