Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Jacobi Prior: An Alternative Bayesian Method for Supervised Learning

Sourish Das, Shouvik Sardar

TL;DR

It is established that the Jacobi estimator is asymptotically close to, and asymptotically equivalent to, the posterior mode under the Jacobi prior, and a parallelisable Monte Carlo algorithm is proposed to quantify the uncertainty in the estimated coefficients.

Abstract

The Jacobi prior offers an alternative Bayesian framework, designed to achieve superior computational efficiency without compromising predictive performance. Compared to widely used methods such as Lasso, Ridge, Elastic Net, uniLasso, the MCMC-based Horseshoe prior, and non-Bayesian machine learning methods including Support Vector Machines (SVM), Random Forests, and Extreme Gradient Boosting (XGBoost), the Jacobi prior achieves competitive or better accuracy with significantly reduced computational cost. The method is well suited to distributed computing environments, as it naturally accommodates partitioned data across multiple servers. We propose a parallelisable Monte Carlo algorithm to quantify the uncertainty in the estimated coefficients. We establish that the Jacobi estimator is asymptotically close to, and asymptotically equivalent to, the posterior mode under the Jacobi prior. To demonstrate its practical utility, we conduct a comprehensive simulation study comprising seven experiments focused on statistical consistency, prediction accuracy, scalability, sensitivity analysis and robustness study. We further present three real-data applications multi-class classification of stars, quasars, and galaxies using Sloan Digital Sky Survey data, and spinal degeneration classification using sagittal MRI scans from the RSNA 2024 Lumbar Spine Degenerative Classification Challenge. In the spine classification task, we extract last-layer features from a fine-tuned ResNet-50 model and evaluate multiple classifiers, including Jacobi-Multinomial logit regression, SVM, and Random Forest. All code and datasets used in this paper are available at: https://github.com/sourish-cmi/Jacobi-Prior/

Jacobi Prior: An Alternative Bayesian Method for Supervised Learning

TL;DR

It is established that the Jacobi estimator is asymptotically close to, and asymptotically equivalent to, the posterior mode under the Jacobi prior, and a parallelisable Monte Carlo algorithm is proposed to quantify the uncertainty in the estimated coefficients.

Abstract

The Jacobi prior offers an alternative Bayesian framework, designed to achieve superior computational efficiency without compromising predictive performance. Compared to widely used methods such as Lasso, Ridge, Elastic Net, uniLasso, the MCMC-based Horseshoe prior, and non-Bayesian machine learning methods including Support Vector Machines (SVM), Random Forests, and Extreme Gradient Boosting (XGBoost), the Jacobi prior achieves competitive or better accuracy with significantly reduced computational cost. The method is well suited to distributed computing environments, as it naturally accommodates partitioned data across multiple servers. We propose a parallelisable Monte Carlo algorithm to quantify the uncertainty in the estimated coefficients. We establish that the Jacobi estimator is asymptotically close to, and asymptotically equivalent to, the posterior mode under the Jacobi prior. To demonstrate its practical utility, we conduct a comprehensive simulation study comprising seven experiments focused on statistical consistency, prediction accuracy, scalability, sensitivity analysis and robustness study. We further present three real-data applications multi-class classification of stars, quasars, and galaxies using Sloan Digital Sky Survey data, and spinal degeneration classification using sagittal MRI scans from the RSNA 2024 Lumbar Spine Degenerative Classification Challenge. In the spine classification task, we extract last-layer features from a fine-tuned ResNet-50 model and evaluate multiple classifiers, including Jacobi-Multinomial logit regression, SVM, and Random Forest. All code and datasets used in this paper are available at: https://github.com/sourish-cmi/Jacobi-Prior/
Paper Structure (27 sections, 3 theorems, 90 equations, 6 figures, 11 tables, 1 algorithm)

This paper contains 27 sections, 3 theorems, 90 equations, 6 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Spine Degeneration Classification
  3. Motivation for Scalable Bayesian Methods
  4. Jacobi Prior
  5. Jacobi prior on $\hbox{\boldmath $\beta$}$
  6. Jacobi Estimator using the Projection
  7. Parallaisable Monte Carlo Algorithm to Estimate Uncertainty
  8. Asymptotic Properties of Jacobi Estimator
  9. Solution for Predictive Models with Jacobi Prior
  10. Logit Regression
  11. Probit Regression
  12. Poisson Regression
  13. Distributed Multinomial Regression
  14. Learning from Partitioned Data in Distributed Systems
  15. Jacobi Prior for the Gaussian Process Models
  16. ...and 12 more sections

Key Result

Theorem 2.1

Suppose $\pi(\hbox{\boldmath $\eta$})$ is the prior distribution on $\hbox{\boldmath $\eta$}$, where $\hbox{\boldmath $\eta$}=(\eta_1,\eta_2,\cdots,\eta_n).$ Consider $\hbox{\boldmath $\beta$}=A\hbox{\boldmath $\eta$}$, where $\hbox{\boldmath $\beta$}$ is a $p$-dimension vector and $A$ represents $p

Figures (6)

  • Figure 1: Representative MRI slices from the RSNA 2024 Lumbar Spine Degenerative Classification dataset. (a) A sagittal T2-weighted DICOM image showing the lumbar vertebrae, intervertebral discs, and spinal canal in longitudinal view. This view is commonly used to assess disc height loss and spinal stenosis. (b) An axial DICOM image providing a transverse cross-section of the spinal canal and surrounding structures. This view is useful for identifying lateral recess and foraminal narrowing. Together, these views support comprehensive evaluation of lumbar spine degeneration.
  • Figure 2: Figures (a) and (b) show the simulated data and estimated latent sinc function for Example (\ref{['example_sinc_function']}). Figures (c) and (d) display the simulated data and estimated circular decision boundary for Example (\ref{['example_circular_class']}).
  • Figure 3: Figure presents the RMSE of $\hbox{\boldmath $\beta$}$ of Logistic Regression with increasing sample size ($n$), when (a) $a=b=1/2$; and (b) $a=b=1/n$
  • Figure 4: Sensitivity Analysis of Jacobi Estimator for logistic regression. RMSE over grid of (a,b) is being presented, where a and b are hyper-parameters.
  • Figure 5: A graphical presentation of SDSS data to visualise different features and the type of astronomical object.
  • ...and 1 more figures

Theorems & Definitions (15)

  • Theorem 2.1
  • proof
  • Theorem 2.2
  • Remark 2.1
  • Remark 2.2
  • Theorem 2.3
  • proof
  • Remark 3.1
  • Remark 4.1
  • Remark 4.2
  • ...and 5 more