Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enriching Disentanglement: From Logical Definitions to Quantitative Metrics

Yivan Zhang, Masashi Sugiyama

TL;DR

This work establishes algebraic relationships between logical definitions and quantitative metrics to derive theoretically grounded disentanglement metrics and empirically demonstrates the effectiveness of the proposed metrics by isolating different aspects of disentangled representations.

Abstract

Disentangling the explanatory factors in complex data is a promising approach for generalizable and data-efficient representation learning. While a variety of quantitative metrics for learning and evaluating disentangled representations have been proposed, it remains unclear what properties these metrics truly quantify. In this work, we establish algebraic relationships between logical definitions and quantitative metrics to derive theoretically grounded disentanglement metrics. Concretely, we introduce a compositional approach for converting a higher-order predicate into a real-valued quantity by replacing (i) equality with a strict premetric, (ii) the Heyting algebra of binary truth values with a quantale of continuous values, and (iii) quantifiers with aggregators. The metrics induced by logical definitions have strong theoretical guarantees, and some of them are easily differentiable and can be used as learning objectives directly. Finally, we empirically demonstrate the effectiveness of the proposed metrics by isolating different aspects of disentangled representations.

Enriching Disentanglement: From Logical Definitions to Quantitative Metrics

TL;DR

This work establishes algebraic relationships between logical definitions and quantitative metrics to derive theoretically grounded disentanglement metrics and empirically demonstrates the effectiveness of the proposed metrics by isolating different aspects of disentangled representations.

Abstract

Disentangling the explanatory factors in complex data is a promising approach for generalizable and data-efficient representation learning. While a variety of quantitative metrics for learning and evaluating disentangled representations have been proposed, it remains unclear what properties these metrics truly quantify. In this work, we establish algebraic relationships between logical definitions and quantitative metrics to derive theoretically grounded disentanglement metrics. Concretely, we introduce a compositional approach for converting a higher-order predicate into a real-valued quantity by replacing (i) equality with a strict premetric, (ii) the Heyting algebra of binary truth values with a quantale of continuous values, and (iii) quantifiers with aggregators. The metrics induced by logical definitions have strong theoretical guarantees, and some of them are easily differentiable and can be used as learning objectives directly. Finally, we empirically demonstrate the effectiveness of the proposed metrics by isolating different aspects of disentangled representations.
Paper Structure (62 sections, 19 theorems, 90 equations, 14 figures, 11 tables)

This paper contains 62 sections, 19 theorems, 90 equations, 14 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Disentangled representation learning: definitions, metrics, and methods
  3. A logical and algebraic approach to defining and measuring disentangled representations
  4. Contributions
  5. Logical definitions of disentangled representations
  6. Informativeness: injectivity or retractability of a learning model
  7. Modularity: product structure preserved by a learning model
  8. Enrichment: from logic to metric
  9. From equality predicate to strict premetric
  10. From logical operation to quantitative operation
  11. From compound predicate to compound quantity
  12. (Sub)homomorphism from metric to logic
  13. Quantitative metrics of disentangled representations
  14. Modularity metrics via product approximation
  15. Modularity metrics via constancy
  16. ...and 47 more sections

Key Result

Theorem 1

Let $p: A \to {\set{\top, \bot}}$ be a predicate on a set $A$, and let $q: A \to {[0, \infty]}$ be a quantity converted from $p$ according to tab:conversion. Then, for any $a \in A$, $q(a) = 0$ implies $p(a) = \top$. Conversely, for any $a \in A$, $p(a) = \top$ implies $q(a) = 0$ if and only if $p$

Figures (14)

  • Figure 1: Disentangled representation learning
  • Figure 2: From predicates and logical operations to quantities and quantitative operations
  • Figure 3: Negation
  • Figure 4: Conjunction
  • Figure 5: Disjunction
  • ...and 9 more figures

Theorems & Definitions (78)

  • Definition 1: Injective function
  • Definition 2: Retractable function
  • Definition 3: Product function
  • Example 1
  • Definition 4: Strict premetric
  • Theorem 1
  • Definition 5: Zero predicate
  • Definition 6: (Sub)homomorphism from a quantity to a predicate
  • Definition 7: (Sub)homomorphism from a quantitative operation to a logical operation
  • Definition 8: (Sub)homomorphism from an aggregator to a quantifier
  • ...and 68 more