Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A kernel-based framework for learning graded relations from data

Willem Waegeman, Tapio Pahikkala, Antti Airola, Tapio Salakoski, Michiel Stock, Bernard De Baets

TL;DR

A general kernel-based framework for learning relations from data is introduced here and is demonstrated through various experiments on synthetic and real-world data, indicating that incorporating domain knowledge about relations improves the predictive performance.

Abstract

Driven by a large number of potential applications in areas like bioinformatics, information retrieval and social network analysis, the problem setting of inferring relations between pairs of data objects has recently been investigated quite intensively in the machine learning community. To this end, current approaches typically consider datasets containing crisp relations, so that standard classification methods can be adopted. However, relations between objects like similarities and preferences are often expressed in a graded manner in real-world applications. A general kernel-based framework for learning relations from data is introduced here. It extends existing approaches because both crisp and graded relations are considered, and it unifies existing approaches because different types of graded relations can be modeled, including symmetric and reciprocal relations. This framework establishes important links between recent developments in fuzzy set theory and machine learning. Its usefulness is demonstrated through various experiments on synthetic and real-world data.

A kernel-based framework for learning graded relations from data

TL;DR

A general kernel-based framework for learning relations from data is introduced here and is demonstrated through various experiments on synthetic and real-world data, indicating that incorporating domain knowledge about relations improves the predictive performance.

Abstract

Driven by a large number of potential applications in areas like bioinformatics, information retrieval and social network analysis, the problem setting of inferring relations between pairs of data objects has recently been investigated quite intensively in the machine learning community. To this end, current approaches typically consider datasets containing crisp relations, so that standard classification methods can be adopted. However, relations between objects like similarities and preferences are often expressed in a graded manner in real-world applications. A general kernel-based framework for learning relations from data is introduced here. It extends existing approaches because both crisp and graded relations are considered, and it unifies existing approaches because different types of graded relations can be modeled, including symmetric and reciprocal relations. This framework establishes important links between recent developments in fuzzy set theory and machine learning. Its usefulness is demonstrated through various experiments on synthetic and real-world data.

Paper Structure

This paper contains 16 sections, 8 theorems, 48 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. General framework
  3. Notation and basic concepts
  4. Learning arbitrary relations
  5. Special relations
  6. Reciprocal relations
  7. Symmetric relations
  8. Relationships with fuzzy set theory
  9. Relationships with other machine learning algorithms
  10. Experiments
  11. Synthetic data: learning similarity measures
  12. Learning the similarity between documents
  13. Competition between species
  14. Conclusion
  15. Formal definitions
  16. ...and 1 more sections

Key Result

Theorem 2.1

Let us assume that the space of nodes $\mathcal{V}$ is a compact metric space. If a continuous kernel $K^\phi$ is universal on $\mathcal{V}$, then $K_{\otimes}^\Phi$ defines a universal kernel on $\mathcal{E}$.

Figures (3)

  • Figure 1: Left: example of a multi-graph representing the most general case, where no additional properties of relations are assumed. Right: examples of eight different types of relations in a graph of cardinality three. The following relational properties are illustrated: (C) crisp, (G) graded, (R) reciprocal, (S) symmetric, (T) transitive and (I) intransitive. For the reciprocal relations, (I) refers to a relation that does not satisfy weak stochastic transitivity, while (T) is showing an example of a relation fulfilling strong stochastic transitivity. For the symmetric relations, (I) refers a relation that does not satisfy $T$-transitivity w.r.t. the Ł ukasiewicz t-norm $T_{\bf L}(a,b) = \max(a+b-1,0)$, while (T) is showing an example of a relation that fulfills $T$-transitivity w.r.t. the product t-norm $T_{\bf P}(a,b) = ab$. See Section 4 for formal definitions of transitivity.
  • Figure 2: The distribution of similarity scores obtained on a 100 by 100 matrix for all three members of the family. From top to bottom: $(t,t',u,v) = (0,1,2,2)$, $(t,t',u,v) = (0,1,1,0)$ and $(t,t',u,v) = (1,2,1,1)$.
  • Figure 3: The comparison of the ordinary Kronecker product pairwise kernel $K_{\otimes}^{\Phi}$ and the symmetric Kronecker product pairwise kernel $K_{\otimes S}^{\Phi}$ on the Newsgroups dataset. The mean squared error is shown as a function of the training set size.

Theorems & Definitions (21)

  • Theorem 2.1
  • Definition 3.1
  • Definition 3.2
  • Proposition 3.3
  • Theorem 3.4
  • Definition 3.5
  • Definition 3.6
  • Proposition 3.7
  • Theorem 3.8
  • Definition 4.1
  • ...and 11 more