Universitas Muhammadiyah Sidoarjo
Login


Academia Open

 Submit manuscript
Section Physics

A Comparison Some Penalized Variable Selection Methods For Analysis General Linear Regression Model With Application

Vol. 11 No. 2 (2026): December:
DOI : https://doi.org/10.21070/acopen.11.2026.15042
Published : Jul 15, 2026

Tareq Azeez Salih (1), Ali Hameed Yousif (2), Zainab Kadhum Mezher (3)

(1) College of Administration and Economic, Wasit University, Iraq
(2) College of Administration and Economic, Wasit University, Iraq
(3) College of Administration and Economic, Wasit University, Iraq
Fulltext View | Download View | Download

Abstract:

General Background The general linear regression model balances nonparametric flexibility with parametric explanatory power. Specific Background High-dimensional data requires robust regularization to isolate critical predictors. Knowledge Gap Classical approaches falter with small datasets or multicollinearity, triggering the curse of dimensionality. Aims This study compares smoothly clipped absolute deviation methods and the mini-max concave penalty approach for simultaneous parameter estimation and covariate selection. Results Simulations across various sample sizes and correlation intensities show that the SCAD-L2 penalty consistently achieves the lowest average error. Novelty Empirical implementation using clinical records from Al-Kut Teaching Hospital confirms SCAD-L2 isolates critical variables by driving non-significant coefficients to zero. Implications This optimized methodology enhances predictive accuracy and simplifies model architectures without losing explanatory control.


Keywords: General Linear Regression, Variable Selection, Penalty Function, SCAD-L2 Method, MCP Method
 
Key Findings Highlights
SCAD-L2 consistently outperforms alternative regularization methods across diverse sample sizes and high correlation configurations.
Empirical medical database validation successfully isolated critical blood urea determinants by eliminating non-significant clinical indicators.
The integrated mathematical algorithm achieves stable parameter convergence while minimizing the overall mean absolute error.

Downloads

Download data is not yet available.

References

Florentine Bunea et al., "Penalized least squares regression methods and applications to neuroimaging," NeuroImage, vol. 55, pp. 1519-1527, 2011.

J. L. Horowitz, "Semi parametric models," Department of Economics, Northwestern University, U.S.A., Working Paper No. 17, 2004.

J. Liu, R. Zhang, W. Zhao, and Y. Lu, "A robust and efficient estimation method for single index models," Journal of Multivariate Analysis, vol. 122, pp. 226-238, 2013.

K. Jiratchayat and C. Bumrungsup, "Penalized Linear Regression Methods where the Predictors Have Grouping Effect," Thailand Statistician, vol. 17, no. 2, pp. 212-222, 2019.

M. Lu et al., "Application of penalized linear regression methods to the selection of environmental naturopathy," Biometrika: Biomarker Research, vol. 5, p. 9, 2017.

O. Akkus, "Xplore Package for The Popular Parametric and The semi-parametric single index models," Gazi University Journal of Science, vol. 24, no. 4, pp. 753-762, 2011.

Q. Wang and W. Yao, "An adaptive estimation of MAVE," Journal of Multivariate Analysis, vol. 104, pp. 88-100, 2012.

St. Johns, "Variable selection in multivariate Multiple Regression," M.S. thesis, Dept. Math. Stat., Memorial University, 2015.

T. Wang, P. Xu, and L. Zhu, "Penalized minimum Average variance estimation," Statistica Sinica, vol. 23, pp. 543-569, 2013.

X. Liu, "Penalized variable selection for semi-parametric Regression models," Ph.D. dissertation, University of Rochester, New York, 2011.

X. Guo, T. Wang, and L. Zhu, "Model checking for parametric single index models: a dimension reduction model adaptive approach," Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 78, no. 5, pp. 1013-1035, 2016.

Y. Xia, H. Tong, W. K. Li, and L. Zhu, "An adaptive estimation of dimension reduction space," Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 64, no. 3, pp. 363-410, 2002.

Y. Xia, W. Härdle, and O. Linton, "Optimal smoothing for a computationally and statistically Efficient single index Estimator," SFB 649 Discussion Papers, No. 2009-028, Humboldt University, Berlin, Germany, 2009.

Y. Xia and W. Härdle, "Semi-parametric estimation of partially linear single index models," Journal of Multivariate Analysis, vol. 97, pp. 1162-1184, 2006.

L. Zeng and J. Xie, "Group variable selection via SCAD-L2," Statistics: A Journal of Theoretical and Applied Statistics, vol. 48, no. 1, pp. 49-66, 2012.