Ridge Estimation's Effectiveness for Multiple Linear Regression with Multicollinearity: An Investigation Using Monte-Carlo Simulations
Keywords:Ridge Estimation, Multicollinearity, Monte-Carlo, Simulations
The goal of this research is to compare multiple linear regression coefficient estimations with multicollinearity. In order to quantify the effectiveness of estimations by the mean of average mean square error, the ordinary least squares technique (OLS), modified ridge regression method (MRR), and generalized Liu-Kejian method (LKM) are compared (AMSE). For this study, the simulation scenarios are 3 and 5 independent variables with zero mean normally distributed random error of variance 1, 5, and 10, three correlation coefficient levels; i.e., low (0.2), medium (0.5), and high (0.8) are determined for independent variables, and all combinations are performed with sample sizes 15, 55, and 95 by Monte Carlo simulation technique for 1,000 times in total. As the sample size rose, the AMSE decreased. The MRR and LKM both outperformed the LSM. At random error of variance 10, the MRR is the most suitable for all circumstances.
K. K. Adesanya, A. I. Taiwo, A. F. Adedotun & T. O. Olatayo “Modeling Continuous Non-Linear Data with Lagged Fractional Polynomial Regression”, Asian Journal of Applied Sciences 06 (2018) 315. DOI: https://doi.org/10.24203/ajas.v6i5.5492
G. Ciulla, & A. D’Amico “Building energy performance forecasting: Amultiple linear regression approach”, Applied Energy 253 (2019) 113500. DOI: https://doi.org/10.1016/j.apenergy.2019.113500
H. Yang, & X. Chang “A new two-parameter estimator in linear regression”, Communications in Statistics - Theory and Methods 39 (2010) 923 doi: 10.1080/0361092090280791. DOI: https://doi.org/10.1080/03610920902807911
B. M. G. Kibria “Performance of some new ridge regression estimators”, Communications in Statistics - Simulation and Computation, 32 (2003) 419. doi: 10.1081/SAC-120017499. DOI: https://doi.org/10.1081/SAC-120017499
Y. Li, & H. Yang “On the performance of the jackknifed modified ridge estimator in the linear regression model with correlated or heteroscedastic errors”, Communications in Statistics - Theory and Methods 40 (2011) 2695. doi: 10.1080/03610926.2010.491589 DOI: https://doi.org/10.1080/03610926.2010.491589
D. C. Montgomery, E. A. Peck E & G. G. Vining “Introduction to Linear Regression Analysis”, (United States: John Wiley & Sons) 2001.
M. Giacalone, D. Panarello, & R. Mattera “Multicollinearity in regression: an effciency comparison between L p-norm and least squares estimators”, Quality & Quantity 52 (2018) 1831. DOI: https://doi.org/10.1007/s11135-017-0571-y
M. I Ullah, M. Aslam, S. Altaf, & M. Ahmed “Some new diagnostics of multicollinearity in linear regression model”, Sains Malaysiana 48 (2019) 2051. DOI: https://doi.org/10.17576/jsm-2019-4809-26
A. F. Lukman, K. Ayinde, & A. S. Ajiboye “Monte Carlo study of some classification-based ridge parameter estimators”, Journal of Modern Applied Statistical Methods 16 (2017) 24. DOI: https://doi.org/10.22237/jmasm/1493598240
S. H. Choi, H. Y. Jung, & H. Kim, “Ridge fuzzy regression model”, International Journal of Fuzzy Systems 21 (2019) 2077. DOI: https://doi.org/10.1007/s40815-019-00692-0
M. R. Lavery, P. Acharya, S. A. Sivo, & L. Xu, “Number of predictors and multicollinearity: What are their e ects on error and bias in regression”, Communications in Statistics-Simulation and Computation, 48 (2019) 27. DOI: https://doi.org/10.1080/03610918.2017.1371750
M. G. Akbari, & G. Hesamian “A partial-robust-ridge-based regression model with fuzzy predictors-responses”, Journal of Computational and Applied Mathematics 351 (2019) 290. DOI: https://doi.org/10.1016/j.cam.2018.11.006
F. A. O Rumere, S. M. Soemartojo & Y.Widyaningsih, “Restricted Ridge Regression estimator as a parameter estimation in multiple linear regression model for multicollinearity case”, Journal of Physics: Conference Series
A. E. Hoerl & R. W. Kennard “Ridge Regression: Biased Estimation for Nonorthogonal Problems”, American Society for Quality 12 (1970) 44. DOI: https://doi.org/10.1080/00401706.1970.10488634
How to Cite
Copyright (c) 2021 Journal of the Nigerian Society of Physical Sciences
This work is licensed under a Creative Commons Attribution 4.0 International License.
The Journal of the Nigerian Society of Physical Sciences (JNSPS) is published under the Creative Commons Attribution 4.0 (CC BY-NC) license. This license was developed to facilitate open access, namely, it allows articles to be freely downloaded and to be re-used and re-distributed without restriction, as long as the original work is correctly cited. More specifically, anyone may copy, distribute or reuse these articles, create extracts, abstracts, and other revised versions, adaptations or derivative works of or from an article, mine the article even for commercial purposes, as long as they credit the author(s).