Feature Selection and Extraction for Graph Neural Networks
Deepak Bhaskar Acharya, Huaming Zhang
TL;DR
The paper tackles the challenge of high-dimensional node features in graphs by extending Gumbel-Softmax-based feature selection to Graph Neural Networks and introducing a mechanism to rank extracted features. It proposes a feature selection/extraction matrix that reduces the input feature dimensionality while integrating with a two-layer GCN, and it demonstrates that 15-20% of the original features can reach roughly 80-85% of the full-feature accuracy on benchmark datasets. The experiments on Cora, Citeseer, and PubMed illustrate both the effectiveness of selecting features and the utility of ranking extracted features, with top-ranked features retaining more predictive power. This approach offers improved scalability and interpretability for GNNs in semi-supervised node classification tasks on standard benchmarks.
Abstract
Graph Neural Networks (GNNs) have been a latest hot research topic in data science, due to the fact that they use the ubiquitous data structure graphs as the underlying elements for constructing and training neural networks. In a GNN, each node has numerous features associated with it. The entire task (for example, classification, or clustering) utilizes the features of the nodes to make decisions, at node level or graph level. In this paper, (1) we extend the feature selection algorithm presented in via Gumbel Softmax to GNNs. We conduct a series of experiments on our feature selection algorithms, using various benchmark datasets: Cora, Citeseer and Pubmed. (2) We implement a mechanism to rank the extracted features. We demonstrate the effectiveness of our algorithms, for both feature selection and ranking. For the Cora dataset, (1) we use the algorithm to select 225 features out of 1433 features. Our experimental results demonstrate their effectiveness for the same classification problem. (2) We extract features such that they are linear combinations of the original features, where the coefficients for each extracted features are non-negative and sum up to one. We propose an algorithm to rank the extracted features in the sense that when using them for the same classification problem, the accuracy goes down gradually for the extracted features within the rank 1 - 50, 51 - 100, 100 - 150, and 151 - 200.