Natural Learning

TL;DR

Natural Learning (NL) proposes a prototype-theory–driven, highly interpretable classifier that relies on sparse, nearest-prototype reasoning rather than weight-based optimization. By integrating LSH-based nearest-neighbor search with recursive feature pruning, NL identifies core features and two prototype samples per decision, yielding human-friendly rules such as evaluating how a test point relates to close prototypes. Across 17 OpenML binary datasets, NL matches or surpasses several interpretable baselines and remains competitive with some black-box models, while delivering substantially lower model variance and faster predictions due to extreme sparsity. This approach offers meaningful advantages for healthcare and high-dimensional domains where interpretability, robustness to noise, and real-time decisions are critical, though it may be less suited to learning rich representations in vision tasks.

Abstract

We introduce Natural Learning (NL), a novel algorithm that elevates the explainability and interpretability of machine learning to an extreme level. NL simplifies decisions into intuitive rules, like "We rejected your loan because your income, employment status, and age collectively resemble a rejected prototype more than an accepted prototype." When applied to real-life datasets, NL produces impressive results. For example, in a colon cancer dataset with 1545 patients and 10935 genes, NL achieves 98.1% accuracy, comparable to DNNs and RF, by analyzing just 3 genes of test samples against 2 discovered prototypes. Similarly, in the UCI's WDBC dataset, NL achieves 98.3% accuracy using only 7 features and 2 prototypes. Even on the MNIST dataset (0 vs. 1), NL achieves 99.5% accuracy with only 3 pixels from 2 prototype images. NL is inspired by prototype theory, an old concept in cognitive psychology suggesting that people learn single sparse prototypes to categorize objects. Leveraging this relaxed assumption, we redesign Support Vector Machines (SVM), replacing its mathematical formulation with a fully nearest-neighbor-based solution, and to address the curse of dimensionality, we utilize locality-sensitive hashing. Following theory's generalizability principle, we propose a recursive method to prune non-core features. As a result, NL efficiently discovers the sparsest prototypes in O(n^2pL) with high parallelization capacity in terms of n. Evaluation of NL with 17 benchmark datasets shows its significant outperformance compared to decision trees and logistic regression, two methods widely favored in healthcare for their interpretability. Moreover, NL achieves performance comparable to finetuned black-box models such as deep neural networks and random forests in 40% of cases, with only a 1-2% lower average accuracy. The code is available via http://natural-learning.cc.

Figures (6)

• Figure 1: NL's first training iteration on a toy dataset. Step 3 nominates candidates for support vectors, and step 6 tests their typicality; Step 4 implements the principle of "generalizability of features" from prototype theory; Steps 7-9 mimic the "flexibility principle" of prototype theory, allowing NL to switch better sparser prototypes at each iteration.
• Figure 2: Geometric view of natural decision boundaries found by triplets in iterations 1 and 2.
• Figure 3: Public datasets from OpenML for binary classification.
• Figure 4: Performance of NL versus baseline methods in 170 stratified test sets (17 datasets).
• Figure 5: CDD of Accuracy and F-Measure. Horz. line implies no statistical significance at $\alpha=0.01$