Penalization Techniques for Remedying Multicollinearity in Multiple Regression Model

Abstract

Multicollinearity is another important issue in multiple regression. Multicollinearity refers to a situation in which two or more predictor variables are highly linearly correlated, i.e. a linear relationship exists between two or more predictor variables. Penalization is one of the best technique for remedying multicollinearity in a model. In this paper, we used a data (with high possibility of multicollinearity) obtained from previous literature to assess the performance of four penalized regression methods, namely; LASSO, Ridge, Elastic net and SCAD. It shows that, all the penalized regression methods considered have lower mean square error (MSE), than the ordinary least squares regression. It also shows that, penalization techniques can make the predictive accuracy of a model better, by lowering the variability in the measures of a regression coefficients, which shrinks the estimates towards zero.