Knowledge Discovery using Unsupervised Cognition
Alfredo Ibias, Hector Antona, Guillem Ramirez-Miranda, Enric Guinovart
TL;DR
The paper tackles knowledge discovery from high-dimensional data using the Unsupervised Cognition framework (iarga24) by proposing a three-stage pipeline: incremental pattern mining to extract representative patterns, correlation-based feature selection to isolate relevant features, and pattern-guided dimensionality reduction to build a simpler, more interpretable UC model. It then retrains on the reduced feature set and uses the final UC model to extract patterns for exploration, demonstrating improved predictive performance on the TCGA Kidney Cancers dataset, including substantial feature reductions (from $58{,}056$ to about $2{,}067$ features) and up to about a $10 ext{ extpercent}$ improvement in accuracy. The methods are evaluated via three experiments, showing state-of-the-art performance in dimensionality reduction and robust pattern validity, with results indicating meaningful, interpretable patterns and practical gains in accuracy. This work delivers a practical, end-to-end approach for knowledge discovery that leverages the structure of UC representations to produce compact, explainable insights applicable to domains like medicine and analytics.
Abstract
Knowledge discovery is key to understand and interpret a dataset, as well as to find the underlying relationships between its components. Unsupervised Cognition is a novel unsupervised learning algorithm that focus on modelling the learned data. This paper presents three techniques to perform knowledge discovery over an already trained Unsupervised Cognition model. Specifically, we present a technique for pattern mining, a technique for feature selection based on the previous pattern mining technique, and a technique for dimensionality reduction based on the previous feature selection technique. The final goal is to distinguish between relevant and irrelevant features and use them to build a model from which to extract meaningful patterns. We evaluated our proposals with empirical experiments and found that they overcome the state-of-the-art in knowledge discovery.