Particle swarm optimization with Applications to Maximum Likelihood Estimation and Penalized Negative Binomial Regression

TL;DR

This paper proposes Particle Swarm Optimization (PSO) as a robust alternative to traditional optimization routines for parameter estimation in nonstandard distributions. By using basic PSO settings (e.g., $c_1=c_2=2$, $\chi=1$, linearly decreasing inertia) and standard boundary handling, the authors demonstrate PSO's ability to reproduce known results, diagnose identifiability issues, overcome convergence failures in log-binomial regression, and yield superior maximum likelihood estimates for complex models such as the Weibull–G and Exponentiated Exponential–Inverse Weibull (EE–IW) distributions. Key findings include PSO matching or exceeding SAS/R results for WE/EW/EE, identifying redundant parameters in WBXII/BBXII with recast-PSO techniques, delivering effective penalized log-binomial regression in heart-disease prediction, and producing better CDF fits for EE–IW datasets. Overall, PSO shows strong potential as a flexible, transparent, and accessible tool for parameter estimation across diverse statistical problems.

Abstract

General purpose optimization routines such as nlminb, optim (R) or nlmixed (SAS) are frequently used to estimate model parameters in nonstandard distributions. This paper presents Particle Swarm Optimization (PSO), as an alternative to many of the current algorithms used in statistics. We find that PSO can not only reproduce the same results as the above routines, it can also produce results that are more optimal or when others cannot converge. In the latter case, it can also identify the source of the problem or problems. We highlight advantages of using PSO using four examples, where: (1) some parameters in a generalized distribution are unidentified using PSO when it is not apparent or computationally manifested using routines in R or SAS; (2) PSO can produce estimation results for the log-binomial regressions when current routines may not; (3) PSO provides flexibility in the link function for binomial regression with LASSO penalty, which is unsupported by standard packages like GLM and GENMOD in Stata and SAS, respectively, and (4) PSO provides superior MLE estimates for an EE-IW distribution compared with those from the traditional statistical methods that rely on moments.

Figures (6)

• Figure 1: Surface of BBXII Log-Likelihood Profile with Fixed $(\alpha,s,c)$.
• Figure 2: Surface of Log Binomial Regression Likelihood with One Intercept and Covariate.
• Figure 3: Convergent Initial Values for some Non-Convergent Sample.
• Figure 4: Fitted probabilities of log binomial LASSO regression for the male cohort. Various shades indicate distinct fitted probability values - red indicating a relatively low probability. Each data point corresponds to a unique observation, totaling 1622 male participants. As we increase $\rho$, the fitted probabilities become more clustered around higher values which aligns with our objective function penalty term. Since a higher $\rho$ will penalizes $\beta$ towards zero,the predicted probabilities tend to one.
• Figure 5: Fitted probabilities of log binomial LASSO regression for the female cohort. Similar to that of the male cohort, the female cohort's predicted probabilities cluster around higher values as we increase $\rho$. The dots represent the 2034 non-missing females.