SmallML: Bayesian Transfer Learning for Small-Data Predictive Analytics

TL;DR

SmallML addresses the SME data scarcity problem by integrating three methodological strands—transfer learning, hierarchical Bayesian inference, and conformal prediction—to deliver enterprise-grade predictive analytics on datasets with as few as 50-200 observations per SME. A SHAP-based transfer learning step extracts informative priors from 22{,}673 public records, which are then refined through cross-SME partial pooling to yield robust SME-specific posteriors. Conformal prediction provides distribution-free finite-sample uncertainty guarantees that complement Bayesian credible intervals, enabling risk-aware decisions in resource-constrained settings. The framework achieves a mean AUC of $0.967 \pm 0.042$ on synthetic SME churn data, with 92% empirical coverage at a 90% target and a training time around 33 minutes on CPU hardware, demonstrating practical feasibility and meaningful democratization of AI capabilities for the roughly 33 million US SMEs.

Abstract

Small and medium-sized enterprises (SMEs) represent 99.9% of U.S. businesses yet remain systematically excluded from AI due to a mismatch between their operational scale and modern machine learning's data requirements. This paper introduces SmallML, a Bayesian transfer learning framework achieving enterprise-level prediction accuracy with datasets as small as 50-200 observations. We develop a three-layer architecture integrating transfer learning, hierarchical Bayesian modeling, and conformal prediction. Layer 1 extracts informative priors from 22,673 public records using a SHAP-based procedure transferring knowledge from gradient boosting to logistic regression. Layer 2 implements hierarchical pooling across J=5-50 SMEs with adaptive shrinkage, balancing population patterns with entity-specific characteristics. Layer 3 provides conformal sets with finite-sample coverage guarantees P(y in C(x)) >= 1-alpha for distribution-free uncertainty quantification. Validation on customer churn data demonstrates 96.7% +/- 4.2% AUC with 100 observations per business -- a +24.2 point improvement over independent logistic regression (72.5% +/- 8.1%), with p < 0.000001. Conformal prediction achieves 92% empirical coverage at 90% target. Training completes in 33 minutes on standard CPU hardware. By enabling enterprise-grade predictions for 33 million U.S. SMEs previously excluded from machine learning, SmallML addresses a critical gap in AI democratization. Keywords: Bayesian transfer learning, hierarchical models, conformal prediction, small-data analytics, SME machine learning

Figures (2)

• Figure 4.1: SmallML Three-Layer Architecture. The framework addresses small-data challenges through modular integration of transfer learning (Layer 1), hierarchical Bayesian modeling (Layer 2), and conformal prediction (Layer 3). The visual pipeline shows data flow: transfer learning extracts priors ($\bm{\beta}_0$, $\bm{\Sigma}_0$) from large public data, hierarchical Bayesian inference pools strength across $J$ SMEs, and conformal calibration provides distribution-free uncertainty.
• Figure 4.2: Hierarchical Bayesian Model Plate Notation. The three-level structure shows: (1) population hyperparameters $\bm{\mu}$, $\sigma$ informed by transfer learning priors $\bm{\beta}_0$, $\bm{\Sigma}_0$; (2) SME-specific coefficients $\bm{\beta}_j$ for $j=1,\ldots,J$ SMEs; (3) customer churn observations $y_{ij}$ for $i=1,\ldots,n_j$ customers per SME. Nested plates indicate repeated structures enabling partial pooling across SMEs. Arrows indicate conditional dependencies, with transfer priors shown as fixed hyperparameter inputs. Observations are conditional on feature vectors $\mathbf{x}_{ij}$ (customer covariates), and the logistic function is $\sigma(z) = 1/(1+e^{-z})$.