Ridge Estimation's Effectiveness for Multiple Linear Regression with Multicollinearity: An Investigation Using Monte-Carlo Simulations

O. G. Obadina; Adedayo Funmi Adedotuun; O. A. Odusanya

doi:10.46481/jnsps.2021.304

Authors

O. G. Obadina
Department of Mathematical Sciences, Olabisi Onabanjo University, Ago-Iwoye, Ogun State, Nigeria
Adedayo Funmi Adedotuun
[email protected]

Department of Mathematics, Covenant University Ota, Ogun State, Nigeria
O. A. Odusanya
Department of Mathematics, D.S Adegbenro (ICT) Polytechnic, Itori, Ogun State, Nigeria

Keywords:

Ridge Estimation, Multicollinearity, Monte-Carlo, Simulations

Abstract

The goal of this research is to compare multiple linear regression coefficient estimations with multicollinearity. In order to quantify the effectiveness of estimations by the mean of average mean square error, the ordinary least squares technique (OLS), modified ridge regression method (MRR), and generalized Liu-Kejian method (LKM) are compared (AMSE). For this study, the simulation scenarios are 3 and 5 independent variables with zero mean normally distributed random error of variance 1, 5, and 10, three correlation coefficient levels; i.e., low (0.2), medium (0.5), and high (0.8) are determined for independent variables, and all combinations are performed with sample sizes 15, 55, and 95 by Monte Carlo simulation technique for 1,000 times in total. As the sample size rose, the AMSE decreased. The MRR and LKM both outperformed the LSM. At random error of variance 10, the MRR is the most suitable for all circumstances.

Dimensions

REFERENCES

K. K. Adesanya, A. I. Taiwo, A. F. Adedotun & T. O. Olatayo “Modeling Continuous Non-Linear Data with Lagged Fractional Polynomial Regression”, Asian Journal of Applied Sciences 06 (2018) 315.

G. Ciulla, & A. D’Amico “Building energy performance forecasting: Amultiple linear regression approach”, Applied Energy 253 (2019) 113500.

H. Yang, & X. Chang “A new two-parameter estimator in linear regression”, Communications in Statistics - Theory and Methods 39 (2010) 923 doi: 10.1080/0361092090280791.

B. M. G. Kibria “Performance of some new ridge regression estimators”, Communications in Statistics - Simulation and Computation, 32 (2003) 419. doi: 10.1081/SAC-120017499.

Y. Li, & H. Yang “On the performance of the jackknifed modified ridge estimator in the linear regression model with correlated or heteroscedastic errors”, Communications in Statistics - Theory and Methods 40 (2011) 2695. doi: 10.1080/03610926.2010.491589

D. C. Montgomery, E. A. Peck E & G. G. Vining “Introduction to Linear Regression Analysis”, (United States: John Wiley & Sons) 2001.

M. Giacalone, D. Panarello, & R. Mattera “Multicollinearity in regression: an effciency comparison between L p-norm and least squares estimators”, Quality & Quantity 52 (2018) 1831.

M. I Ullah, M. Aslam, S. Altaf, & M. Ahmed “Some new diagnostics of multicollinearity in linear regression model”, Sains Malaysiana 48 (2019) 2051.

A. F. Lukman, K. Ayinde, & A. S. Ajiboye “Monte Carlo study of some classification-based ridge parameter estimators”, Journal of Modern Applied Statistical Methods 16 (2017) 24.

S. H. Choi, H. Y. Jung, & H. Kim, “Ridge fuzzy regression model”, International Journal of Fuzzy Systems 21 (2019) 2077.

M. R. Lavery, P. Acharya, S. A. Sivo, & L. Xu, “Number of predictors and multicollinearity: What are their e ects on error and bias in regression”, Communications in Statistics-Simulation and Computation, 48 (2019) 27.

M. G. Akbari, & G. Hesamian “A partial-robust-ridge-based regression model with fuzzy predictors-responses”, Journal of Computational and Applied Mathematics 351 (2019) 290.

F. A. O Rumere, S. M. Soemartojo & Y.Widyaningsih, “Restricted Ridge Regression estimator as a parameter estimation in multiple linear regression model for multicollinearity case”, Journal of Physics: Conference Series

(2021) 1725.

A. E. Hoerl & R. W. Kennard “Ridge Regression: Biased Estimation for Nonorthogonal Problems”, American Society for Quality 12 (1970) 44.