Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intrinsic Concept Extraction Based on Compositional Interpretability

Hanyu Shi, Hong Tao, Guoheng Huang, Jianbin Jiang, Xuhang Chen, Chi-Man Pun, Shanhu Wang, Pan Pan

Abstract

Unsupervised Concept Extraction aims to extract concepts from a single image; however, existing methods suffer from the inability to extract composable intrinsic concepts. To address this, this paper introduces a new task called Compositional and Interpretable Intrinsic Concept Extraction (CI-ICE). The CI-ICE task aims to leverage diffusion-based text-to-image models to extract composable object-level and attribute-level concepts from a single image, such that the original concept can be reconstructed through the combination of these concepts. To achieve this goal, we propose a method called HyperExpress, which addresses the CI-ICE task through two core aspects. Specifically, first, we propose a concept learning approach that leverages the inherent hierarchical modeling capability of hyperbolic space to achieve accurate concept disentanglement while preserving the hierarchical structure and relational dependencies among concepts; second, we introduce a concept-wise optimization method that maps the concept embedding space to maintain complex inter-concept relationships while ensuring concept composability. Our method demonstrates outstanding performance in extracting compositionally interpretable intrinsic concepts from a single image.

Intrinsic Concept Extraction Based on Compositional Interpretability

Abstract

Unsupervised Concept Extraction aims to extract concepts from a single image; however, existing methods suffer from the inability to extract composable intrinsic concepts. To address this, this paper introduces a new task called Compositional and Interpretable Intrinsic Concept Extraction (CI-ICE). The CI-ICE task aims to leverage diffusion-based text-to-image models to extract composable object-level and attribute-level concepts from a single image, such that the original concept can be reconstructed through the combination of these concepts. To achieve this goal, we propose a method called HyperExpress, which addresses the CI-ICE task through two core aspects. Specifically, first, we propose a concept learning approach that leverages the inherent hierarchical modeling capability of hyperbolic space to achieve accurate concept disentanglement while preserving the hierarchical structure and relational dependencies among concepts; second, we introduce a concept-wise optimization method that maps the concept embedding space to maintain complex inter-concept relationships while ensuring concept composability. Our method demonstrates outstanding performance in extracting compositionally interpretable intrinsic concepts from a single image.
Paper Structure (19 sections, 2 theorems, 20 equations, 5 figures, 3 tables)

This paper contains 19 sections, 2 theorems, 20 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generative Concept Learning
  4. Hyperbolic learning
  5. Compositionality in Concept Extraction
  6. Preliminaries
  7. Diffusion Models
  8. Hyperbolic Geometry
  9. Concepts and Compositionality
  10. Method
  11. Concept Learning
  12. Concept-wise Optimization
  13. Overall training objective
  14. Experiments
  15. Baseline and Evaluation Metrics
  16. ...and 4 more sections

Key Result

Proposition 1

For concept tokens $[V_i],[V_j] \in \mathcal{T}$, the concept representation $R:\mathcal{T} \to \mathcal{V}$ is considered compositional if there exist positive weights $w_i,w_i \in \mathbb{R}^+$ such that:

Figures (5)

  • Figure 1: The difference between unsupervised concept extraction (UCE) methods Stein2024cendra2025ICEAutoConcept and the composable and interpretable intrinsic concept extraction method. Object-level concept extraction method AutoConcepthao2024conceptexpress can only extract object-level concepts and is unable to extract attribute-level concepts such as color and material. Although the intrinsic concept extraction method cendra2025ICE can extract both object-level concepts and attribute-level concepts, the concepts it extracts are not sufficiently close to the original concepts, and it fails to consider the compositionality of the embedding space; thus, it has poor interpretability. In contrast, HyperExpress considers the relationships between concepts when learning them. As a result, the extracted concepts are more aligned with the objects in the image. Additionally, it imposes compositional constraints on the concept embedding space, thereby enabling concept extraction and combination capabilities that are understandable to humans.
  • Figure 2: The Proposed Method and Its Components. (a) The overall structure of HyperExpress: It addresses the CI-ICE task from two aspects: concept learning and concept-wise optimization. (b) Concept learning: It leverages triplet loss $\mathcal{L}_{triplet}$ and hyperbolic entailment loss $\mathcal{L}_{entail}$ to learn the hierarchical structure and associative relationships between object-level concepts and attribute-level concepts. (c) Concept-wise Optimization: It uses Horosphere Projector (HP) to constrain the concept space, thereby ensuring the compositionality of concepts.
  • Figure 3: Explanation of the proposed HEL module. Unlike the HCL module, the hyperbolic entailment loss is computed within the Lorentz model. If the object-level concept ($v_k^{obj}$) and attribute-level concept ($v_k^{color}$ and $v_k^{material}$) satisfy the condition in \ref{['eq:yh']}, the entailment loss will be $0$; otherwise, the corresponding entailment loss will be calculated.
  • Figure 4: Horosphere Projection Module and an illustration of concept compositionality. If no constraints are applied to the concept embedding space, it may lead to incorrect composition of concepts. For instance, using $v_k^{att}+v_k^{obj}$ alone cannot reconstruct the primary concept. To address this, we reproject the embedding space to ensure composability between concepts.
  • Figure 5: Comparison of Qualitative Results Between HyperExpress and ICE cendra2025ICE. This comparison includes two processes: one is extracting object-level concepts and further attribute-level concepts from a single image, and the other is performing compositional reconstruction using the extracted concepts.

Theorems & Definitions (2)

  • Proposition 1
  • Proposition 2