Aligning Knowledge Graphs Provided by Humans and Generated from Neural Networks in Specific Tasks

TL;DR

The paper tackles the challenge of aligning neural-network-generated concept knowledge with human-provided knowledge by introducing a plug-and-play framework that uses Vector Symbolic Architecture to map neural vectors to a concept-level knowledge graph and to align it with a human knowledge graph $KG_G$. It replaces traditional embeddings with a four-dimensional tensor $KGV_{NN}$ and employs two bipolar VSAs to translate between $KG_{NN}$ and $KG_G$, optimizing a composite loss $L = L_{VSA}+L_R+L_T$ that enforces concept alignment while preserving end-to-end training. Key contributions include the novel autoencoder design with $KGV_{NN}$, a vector-based graph alignment via cosine similarity and bipartite matching, and regulators that enforce symbol independence and binary-like value constraints. The experiments on MNIST demonstrate high matching consistency (e.g., average cosine similarity near 1) and reveal how richer human knowledge and larger concept spaces improve alignment, supporting the feasibility of integrated neural-symbolic reasoning and knowledge graph completion within neural models. Overall, the approach enhances interpretability and provides a pathway for incorporating symbolic reasoning into neural systems with potential for discovering useful concepts beyond human-defined knowledge.

Abstract

This paper develops an innovative method that enables neural networks to generate and utilize knowledge graphs, which describe their concept-level knowledge and optimize network parameters through alignment with human-provided knowledge. This research addresses a gap where traditionally, network-generated knowledge has been limited to applications in downstream symbolic analysis or enhancing network transparency. By integrating a novel autoencoder design with the Vector Symbolic Architecture (VSA), we have introduced auxiliary tasks that support end-to-end training. Our approach eschews traditional dependencies on ontologies or word embedding models, mining concepts from neural networks and directly aligning them with human knowledge. Experiments show that our method consistently captures network-generated concepts that align closely with human knowledge and can even uncover new, useful concepts not previously identified by humans. This plug-and-play strategy not only enhances the interpretability of neural networks but also facilitates the integration of symbolic logical reasoning within these systems.

Figures (3)

• Figure 1: (left) The structure of the method, in which $KGV_{NN}$ can be used for downstream tasks, like autoencoder or specific tasks; (right) A sample knowledge graph for MNIST.)
• Figure 2: (a) $VSA_{NN}$ before training, first 50-D; (b) $VSA_{NN}$ after training, first 50-D; (c) Visualization of the average $KGV_{NN}$ for all numbers; (d) Visualization of the average $KGV_{NN}$ for all numbers through the decoder, values $\geq 0.5$; (e) The matching between $VSA_{NN}$ and $VSA_G$.
• Figure 3: Experiment 2 results. All of them show the average cosine similarity of matched vectors in 10 epochs. (a) Takes the number of entities in $KG_G$ as the variable. (b) Takes the number of relations in $KG_G$ as the variable. (c) Takes the number of knowledge (triplets) in $KG_G$ as the variable. (d) Takes the variance of triplets among relations in $KG_G$ as the variable. (e) Takes the number of concepts (entities/relations) in $KG_{NN}$ considering the number of concepts in $KG_G$ as the variable.