English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

It is common for linear regression models to be plagued with the problem of multicollinearity when two or more regressors are highly correlated. This problem results in unstable estimates of regression coefficients and causes some serious problems in validation and interpretation of the model. Different diagnostic measures are used to detect multicollinearity among regressors. Many statistical software and R packages provide few diagnostic measures for the judgment of multicollinearity. Most widely used diagnostic measures in these software are: coefficient of determination (R2), variance inflation factor/tolerance limit (VIF/TOL), eigenvalues, condition number (CN) and condition index (CI) etc. In this manuscript, we present an R package, mctest, that computes popular and widely used multicollinearity diagnostic measures. The package also indicates which regressors may be the reason of collinearity among regressors. Brief introduction of collinearity Consider the conventional multiple linear regression equation y = Xβ + u, where y is an n× 1 vector of observation on response variable, X is known design matrix of order n× p, β is an p× 1 vector of unknown parameters and u is an n× 1 vector of random errors with mean zero and variance σ2 In, where In is an identity matrix of order n. One of the important assumptions of the classical linear regression model is that there is no exact collinearity among the regressors otherwise, the issue is referred to as multicollinearity. Generally, the problem of multicollinearity may also refer to have not only exact linear relationship but also high correlations among some or all regressors of a regression model under study. Strictly speaking, multicollinearity is usually refers to the existence of more than one exact linear relationship among regressors, while collinearity refers to the existence of a single linear relationship among regressors. However, in general, the term multicollinearity may be referred to both the cases. Data collection method, constraints on the fitted regression model, model specification error, overdefined model, some common trend in time series data and naturally correlated data may be some potential sources of multicollinearity. The problem of multicollinearity has potentially serious effect on the regression estimates such as implausible coefficient signs, impossible inversion of matrix X′X as it becomes either singular (in the case of perfect multicollinearity) or near to singular (in the case of near to perfect multicollinearity), large magnitude of coefficients in absolute value, large variance or standard errors with wider confidence intervals and small t-ratios. The ordinary least squared (OLS) estimators and standard errors also become sensitive to small change in data when regressors are collinear to each other (see Belsley et al., 1980; Dorsett et al., 1983; Farrar and Glauber, 1967; Gunst and Mason, 1977; Johnston, 1963; Mason et al., 1975). On the basis of theoretical considerations, these indications signify the need for detection of multicollinearity among regressors (Belsley et al., 1980; Greene, 2002; Younger, 1979). This paper presents the overview of existing collinearity diagnostic measures along with commonly used threshold values for the judgment of existence of collinearity among regressors. These diagnostic measures are being implemented in R with the proposed mctest package (Imdadullah and Aslam, 2016). Collinearity diagnostic measures Several numerical methods for the detection of collinearity are available in the existing literature proposed or discussed by various authors e.g., (see Belsley et al., 1980; Curto and Pinto, 2011; Koutsoyiannis, 1977; Kovács et al., 2005; Marquardt, 1970; Montgomery and Peck, 1982, etc.). Widely used and the most suggested collinearity diagnostic measures are values of pair-wise correlations, R-squared value (c.f. Gujarati and Porter (2008)), variance inflation factor (VIF), tolerance limit (TOL) (Kutner et al., 2004; Marquardt, 1970), eigenvalues (Kendall, 1957; Silvey, 1969), condition number (CN) and condition index (CI) (Belsley et al., 1980), Leamer’s method (Greene, 2002), Kleins rule (Klein, 1962), three tests proposed by Farrar and Glauber (Farrar and Glauber, 1967), the Red indicator (Kovács et al., 2005), and Theil’s measure (Theil, 1971). However, none of these can be regarded as the The R Journal Vol. 8/2, December 2016 ISSN 2073-4859 CONTRIBUTED RESEARCH ARTICLES 496 best choice for the detection of collinearity (Kovács et al., 2005). Following are the diagnostics that can be considered as the classical symptoms of harmfulness of multicollinearity. (i) If zero-order (pair-wise) correlation coefficient between two regressors is high (say >0.8) then multicollinearity may be a serious problem (Gujarati and Porter, 2008; Maddala, 1988). However, it is not sufficient and necessary condition for the detection of multicollinearity because a linear relation involves many of the regressors, therefore it may not be possible to detect such a relation with a simple correlation or pairs-wise plot (Chatterjee and Hadi, 2006; Judge et al., 1985). (ii) High R2 (say >0.8) may indicate the problem of multicollinearity Gujarati and Porter (2008). In most of the cases, overall F-test rejects the null hypothesis of partial slopes for being zero, but some or all individual t-ratios of partial slopes may be non-significant. Therefore, a model having no multicollinearity problem should have high R2 and larger (significant) t-ratios of partial slopes. (iii) High variance of regression coefficients’ estimates and low t-ratios also suggest the existence of multicollinearity. We classified other widely used collinearity diagnostics as overall and individual measures of collinearity. This classification is due to the fact that there are some diagnostic measures resulting in a single number, while other yield as many quantities as the number of regressors in the model. The overall diagnostic measures help to get an idea about only the existence of collinearity and they do not tell which regressor may be the reason of collinearity, while the individual measures point out the regressors causing collinearity. Since no specific collinearity diagnostic measure is superior and each of these measures has different collinearity detection criterion (threshold value) proposed by various authors in the textbooks and research articles, there is need to study multiple collinearity diagnostics. That is, there is no clear-cut criterion for evaluating multicollinearity in linear regression models. Similarly, some diagnostic measures are statistically criticized such as tests proposed by Farrar and Glauber (1967) while threshold values of many other diagnostic measures are subjective in nature as no unique or standard critical values exist for these measures. Moreover, different collinearity detection methods are not comparable with each other. That is why, many regression analysts often rely on more than one collinearity diagnostic measures. Following is the list of overall and individual collinearity diagnostic measures along with short description, formula, detection criterion (threshold value) and reference for each measure. These diagnostic measures will assist the researchers in determining when and where some corrective action is necessary. According to Belsley et al. (1980), the investigations concerning the presence of multicollinearity have been based on judging the magnitudes of various diagnostic measures. Overall collinearity diagnostic measures • Determinant: The matrix X′X will be singular if it contains linearly dependent columns or rows. Therefore, determinant of normalized correlation matrix (R = X′X) without intercept can be used to indicate existence of collinearity among regressors. However, determinant does not provide information about interdependence among regressors, it only provides information about singularity (departure from orthogonality) of a correlation matrix. The determinant of X′X on the scale is 0 ≤ |X′X| ≤ 1 (Cooley and Lohnes, 1971). If |X′X| ∼ 0, then collinearity exists among regressors (Asteriou and Hall, 2007). • R-squared: Coefficient of determination (R2) from regression of all x on y. The R2 is a monotonic nondecreasing function of number of regressors included in the model, that is, R2 indicates how well the regression fits the data (Gujarati and Porter, 2008; Stock and Watson, 2010). On the other hand, higher the R2 values, the more chances of regressors to be plagued with multicollinearity (Asteriou and Hall, 2007; Gujarati and Porter, 2008; Maddala, 1988), since R2 is affected by regressors sharing their variances (Gujarati and Porter, 2008; Maddala, 1988). • Farrar χ2: It is the Chi-square test for detecting the strength of collinearity over the complete set of regressors. χ2 = − [ n− 1− 1 6(2p+5) ] × loge [X′X] ∼ ψ2 v= 1 2 p(p−1) . Collinearity exists among regressors if χ2 > χ1 2 p(p−1) (Farrar and Glauber, 1967). • Condition index: CIj = √ max(λj) λj j = 1, 2, · · · , p; λ1 ≥ λ2 · · · ≥ λp. Collinearity exists if any of CIj > 10, 15, or 30 (Belsley et al., 1980; Chatterjee and Hadi, 2006; Maddala, 1988). • Sum of reciprocal of eigenvalues: In an orthogonal system p ∑ j=1 1 λj = p, therefore, for a sample based correlation matrix R with The R Journal Vol. 8/2, December 2016 ISSN 2073-4859 CONTRIBUTED RESEARCH ARTICLES 497 eigenvalues λj comparing p with p ∑ j=1 1 λ can be used to indicate collinearity. If p ∑ j=1 1 λj is (say) five times larger than the number of regressors used in the model then collinearity exists among regressors (Chatterjee and Price, 1977; Dillon and Goldstein, 1984). • Theil’s indicator: Theil (1971) proposed a measure of collinearity based on an incremental contribution (R2 − Rj ) to the squared multiple correlation, where Rj is the R 2 from auxiliary regression of regressors. m = R2 − p ∑ i=1 (R2 − R−i). If m = 0 then all X’s are mutually uncorrela

mctest: An R Package for Detection of Collinearity among Regressors