Augmented Regression Models using Neurochaos Learning

TL;DR

This work extends regression by incorporating a chaos-inspired feature, the Tracemean $TM$, derived from Neurochaos Learning to augment Linear, Ridge, Lasso, and SVR models. Four augmented regressors—ACLR, ACRR, ACLS, and ACSVR—are evaluated on ten real-world datasets and a synthetic $y=mx+c+\epsilon$ dataset, using $R^2$, MSE, and MAE as metrics, with hyperparameters tuned via cross-validation. The results show that augmentation generally improves predictive performance, with Augmented Chaotic Ridge Regression delivering the largest average boost of about $11.35\%$ and SVR-based methods frequently achieving top $R^2$ scores, while synthetic data confirm convergence of MSE toward MMSE as sample size grows. This work demonstrates the potential of chaos-inspired features to enhance regression accuracy and efficiency, while acknowledging computational costs from hyperparameter tuning and outlining paths for extending the approach with additional chaotic features or parameter-free optimization.

Abstract

This study presents novel Augmented Regression Models using Neurochaos Learning (NL), where Tracemean features derived from the Neurochaos Learning framework are integrated with traditional regression algorithms : Linear Regression, Ridge Regression, Lasso Regression, and Support Vector Regression (SVR). Our approach was evaluated using ten diverse real-life datasets and a synthetically generated dataset of the form $y = mx + c + ε$. Results show that incorporating the Tracemean feature (mean of the chaotic neural traces of the neurons in the NL architecture) significantly enhances regression performance, particularly in Augmented Lasso Regression and Augmented SVR, where six out of ten real-life datasets exhibited improved predictive accuracy. Among the models, Augmented Chaotic Ridge Regression achieved the highest average performance boost (11.35 %). Additionally, experiments on the simulated dataset demonstrated that the Mean Squared Error (MSE) of the augmented models consistently decreased and converged towards the Minimum Mean Squared Error (MMSE) as the sample size increased. This work demonstrates the potential of chaos-inspired features in regression tasks, offering a pathway to more accurate and computationally efficient prediction models.

Figures (7)

• Figure 1: Augmented Chaotic Regression.
• Figure 3: Comparison of $R^2$ Scores between LR and ACLR across Real-World Datasets.
• Figure 4: Comparison of $R^2$ Scores between RR and ACRR across Real-World Datasets.
• Figure 5: Comparison of $R^2$ Scores between LS and ACLS across Real-World Datasets.
• Figure 6: Comparison of $R^2$ Scores between SVR and ACSVR across Real-World Datasets.