Revision: Variance Inflation in Regression

Variance Inflation Factors (VIFs) are reexamined as conditioning diagnostics for models with intercept, with and without centering regressors to their means as oft debated. Conventional VIFs, both centered and uncentered, are flawed. To rectify matters, two types of orthogonality are noted: vector-space orthogonality and uncorrelated centered regressors. The key to our approach lies in feasible Reference models encoding orthogonalities of these types. For models with intercept it is found that (i) uncentered VIFs are not ratios of variances as claimed, owing to infeasible Reference models; (ii) instead they supply informative angles between subspaces of regressors; (iii) centered VIFs are incomplete if not misleading, masking collinearity of regressors with the intercept; and (iv) variance deflation may occur, where ill-conditioned data yield smaller variances than their orthogonal surrogates. Conventional VIFs have all regressors linked, or none, often untenable in practice. Beyond these, our models enable the unlinking of regressors that can be unlinked, while preserving dependence among those intrinsically linked. Moreover, known collinearity indices are extended to encompass angles between subspaces of regressors. To reaccess ill-conditioned data, we consider case studies ranging from elementary examples to data from the literature. 1. Introduction Values for residuals , for , for , for , and for , for . Given of full rank with zero-mean, uncorrelated and homoscedastic errors, the equations yield the estimators for as unbiased with dispersion matrix and . ？？Ill-conditioning, as near dependent columns of , exerts profound and interlinked consequences, causing “crucial elements of to be large and unstable,” “creating inflated variances”; estimates excessive in magnitude, irregular in sign, and “very sensitive to small changes in ”; and unstable algorithms having “degraded numerical accuracy.” See [1–3] for example. Ill-conditioning diagnostics include the condition number , the ratio of its largest to smallest eigenvalues, and the Variance Inflation Factors with , ？that is, ratios of actual ( to “ideal” variances, had the columns of been orthogonal. On scaling the latter to unit lengths and ordering as ,？ is identified in [4] as “the best single measure of the conditioning of the data.” In addition, the bounds of [5] apply also in stepwise regression as in [6–9]. Users deserve to be apprised not only that data are ill conditioned, but also about workings of the diagnostics themselves. Accordingly, we undertake here to rectify long-standing