Representation Synthesis by Probabilistic Many-Valued Logic Operation in Self-Supervised Learning

TL;DR

A logic-operable self-supervised representation learning method by replacing the existing representation synthesis with the OR operation on the probabilistic extension of many-valued logic, which shows that the method performs competitively in single and multi-label classification tasks compared with prior SSL methods using synthetic representations.

Abstract

In this paper, we propose a new self-supervised learning (SSL) method for representations that enable logic operations. Representation learning has been applied to various tasks, such as image generation and retrieval. The logical controllability of representations is important for these tasks. Although some methods have been shown to enable the intuitive control of representations using natural languages as the inputs, representation control via logic operations between representations has not been demonstrated. Some SSL methods using representation synthesis (e.g., elementwise mean and maximum operations) have been proposed, but the operations performed in these methods do not incorporate logic operations. In this work, we propose a logic-operable self-supervised representation learning method by replacing the existing representation synthesis with the OR operation on the probabilistic extension of many-valued logic. The representations comprise a set of feature-possession degrees, which are truth values indicating the presence or absence of each feature in the image, and realize the logic operations (e.g., OR and AND). Our method can generate a representation that has the features of both representations or only those features common to both representations. In addition, the expression of the ambiguous presence of a feature is realized by indicating the feature-possession degree by the probability distribution of truth values of the many-valued logic. We showed that our method performs competitively in single and multi-label classification tasks compared with prior SSL methods using synthetic representations. Moreover, experiments on image retrieval using MNIST and PascalVOC showed that the representations of our method can be operated by OR and AND operations.

Figures (10)

• Figure 1: Overview of the representations and logic operation. The method predicts a representation capable of performing logic operations between representations using many-valued logic. The representation comprises several probability distributions that express the probability of the truth value of the feature-possession degree. These examples show three categorical distributions, possibly corresponding to "Dog", "Person" and "Cat." Logic operations (e.g., OR, AND) between representations can be performed by presenting feature possession in many-valued logic.
• Figure 2: Overview of SSL using mixed images and representation synthesis, such as the logic (OR), mean, and maximum operations.
• Figure 3: Example of the probability of each truth value for $n=4$.
• Figure 4: The-5 images with the highest feature-possession degrees. Feature ID is the index number of feature-possession degrees.
• Figure 5: Boxplot of the feature-possession degrees of particular classes compared with others.