Multicollinearity, or collinearity, occurs when a regression model includes two or more highly related predictors. Peer smoking and perceptions of school smoking norms, for example, are likely to be correlated. If we entered both into a model, we would expect unstable parameter estimates and inflated standard errors. We might find that while either works as a statistically significant predictor of smoking, neither significantly predict smoking when both are included in the same model.
Multicollinearity also occurs in multilevel models when an individuallevel predictor is highly correlated with a grouplevel predictor. Say we created a model that included an individuallevel measure of peer smoking as well as a schoollevel measure of smoking norms and that we created the schoollevel from the aggregate of individual reports of peer smoking. These two measures, at different levels, could be highly correlated, create collinearity, and consequently introduce confounding into the model (Kreft & de Leeuw, 1998).
Collinearity can occur in ordinary least squares regression, maximum likelihood regression, logistic regression, hierarchical linear models, structural equation models, and the like. Below I focus on the effects of collinearity, how it influences completed models, but not so much on detection or remedies.
Correlated independent variables (IVs) create collinearity in a regression model when the relationship is very strong. The result is reduced stability of the corresponding parameter estimates, increased standard errors, and reduced power to measure effects. In the OHT Multilevel papers, we centered our individuallevel predictors on the higherlevel aggregate predictors in part to avoid multicollinearity.
Nearly all sources cited here describe how collinearity can create instability, inflate standard errors, and lower power. Kreft and de Leeuw (1998), for example, talk about how small change in one variable can create a large change in the regression estimates for another. The results of a regression, according to Cohen, Cohen, West, and Aiken (2003), can change considerably by dropping just a few cases when the model contains highly related predictors. The impact depends, however, on the specific model and precisely how it was tested. In testing a taxonomy of models that use R² values or model fit statistics (e.g., AIC), collinearity does not affect results (Burnham & Anderson, 2002; Cohen et al., 2003). This is because collinearity influences the individual parameter estimates and not the overall level of variance accounted for, such as measured by changes in R² values or improvements in fit. These global statistics do not depend on individual parameters and their standard errors. Redundant variables simply fail to improve model fit or R² values because they do not add to the overall prediction.
Harrell (2001) nicely summarizes the issue in the opening to his section on collinearity (§ 4.6): "When at least one of the predictors can be predicted well from the other predictors, the standard error of the regression coefficient estimates can be inflated and the corresponding tests have reduced power. In stepwise variable selection, collinearity can cause predictors to compete and make the selection of 'important' variables arbitrary. Collinearity makes it difficult to estimate and interpret a particular regression coefficient because the data have little information about the effect of changing one variable while holding another (highly correlated) variable constant. However, collinearity does not affect the joint influence of highly correlated variables when tested simultaneously. Therefore, once groups of highly correlated predictors are identified, the problem can be rectified by testing the contribution of an entire set with multiple d.f. test rather than attempting to interpret the coefficient or one d.f. test for a single predictor" (p. 6465).
A careful read of Harrell's collinearity section indicates, in addition, that multicollinearity would influence the standard errors, regression weights, and test statistics of only variables in question. Cohen, Cohen, West, and Aiken (2003) agree. Thus one could enter background variables in a block, even if highly correlated, without influencing the results of later blocks of IVs. This is very important, as it frees the statistician from worrying about relationships in sets of demographic variables, such as between students' academic performance and their parents' educational attainment.
Consider Pierce, Choi, Gilpin, Farkas, and Berry (1998). They present a model that contains familial smoking and peer smoking as well as exposure to tobacco promotions and advertising. Their logistic regression analysis might contain collinearity if peer smoking and familial smoking were highly correlated. Harrell notes that there is "little information about the effects of changing one variable while holding another . . . constant" (p 64). The large sample (N = 1752) in Pierce et al. should, in part, compensate for the collinearity. That is, collinearity is a larger problem in smaller samples given a fixed correlation between IVs. Another way to look at it is that the correlation between offending IVs needs to be greater in larger samples in order to cause the same level of problems.
Collinearity, then, is a problem of degree. There is no single metric to measure collinearity and with the multiple measure, there is no cut point at which experts suddenly consider collinearity a problem. Its effects on individual parameter estimates increase gradually as a function of the increase in relationships among the IVs. Cohen, Cohen, West, and Aiken (2003) give a nice example (Table 10.5.1, p. 421) that demonstrates the effects of collinearity on two variables as their correlation increases from 0.00 to 0.95 with 100 cases. In their example, a correlation as high as 0.50 has little impact on the model, but by the time the correlation between the predictors reaches 0.90, the standard errors over double those in the model with zero correlation. In the Pierce et al. (1998) article, with over 1700 cases, the correlation between familial smoking and peer smoking would need to be larger than 0.50 to have an effect.
Harrell goes on to point out that the effects of collinearity only occur within the set of highly related variables. That is, collinearity between familial smoking and peer smoking in Pierce et al. (1998) affect the estimates for familial smoking and peer smoking, but not the estimates for exposure to tobacco promotions and advertising. Harrell implies that even in the case of substantial collinearity between familial and peer smoking, the estimates for exposure would be unaffected.
Cohen, Cohen, West, and Aiken (2003) give a very nice example of this phenomenon. Below I reproduced Tables 10.5.2 A and 10.5.2 B. The first contains a zero correlation among all four variables and reports the estimate (B), standard error (SE), partial correlation (pr²), the tolerance, and variance inflation factor. The rightmost two columns are standard measures of collinearity. The second contains a correlation among the first three of 0.933, but the fourth remains independent. Note how the estimates and standard errors change substantially for X1 through X3 but the estimates for X4 remain essentially unaffected.
Variable 
B

SE

pr²

Tol

VIF

Intercept 
20.000

0.162
 
X1 
0.387

0.094

0.151

1.000

1.000

X2 
0.391

0.082

0.195

1.000

1.000

X3 
0.400

0.073

0.240

1.000

1.000

X4 
0.391

0.082

0.195

1.000

1.000 
Variable 
B

SE

pr²

Tol

VIF

Intercept 
20.000

0.187
 
X1 
0.086

0.345

0.054

0.099

10.067

X2 
0.137

0.299

0.002

0.099

10.067 
X3 
0.868

0.267

0.100

0.099

10.067 
X4 
0.391

0.094

0.154

1.000

1.000

From this example, any collinearity in familial smoking and peer smoking in the Pierce et al. (1998) model would have little or no impact on the estimates for exposure to tobacco promotions and advertising. I make this claim because 1) Pierce et al. use a much larger sample, 2) if collinearity exists, it is to occur likely between variables other than tobacco promotions and advertising, and 3) correlations between variables modeled by Pierce et al. are not likely to reach high enough levels to cause problems, much greater than r = 0.50.
Nearly every source that talks about multicollinearity contends that it increases the standard errors and that the inflation would be substantial. In the example above, the standard errors triple. Hosmer and Lemeshow (1989) state that "in all cases the tipoff for a problem [of collinearity] comes from the aberrantly large estimated standard errors," which will lead to huge confidence bounds. They give an example where a standard error jumps from 1.0 with one variable in a model to 444.3 with two highly correlated variables in the model. In many cases, then, professional judgment should easily detect excessive standard errors. During the analysis process, there are also other ways to detect collinearity. The tolerance and variance inflation factor in the above tables are examples.
Steps can also be taken to reduce collinearity during analysis if desired. To reduce collinearity, increase the sample size (obtain more data), drop a variable, meancenter or standardize measures, combine variables, or create latent variables. Other options include focusing on changes in fit statistics, such as the AIC or R² values, rather than individual parameter estimates or choosing a different modeling strategy, like partial least squares, ridge, or principal component regression. In some cases, however, retaining highly correlated variables in a block might be preferred (see Harrell, 2001) provided that the analysis includes a large sample and the individual parameter estimates are interpreted with caution.
A complete discussion of all detection and remedies would require considerable time and effort. See the remedies section for the multicolinearity entry in Wikipedia, Cohen, Cohen, West, and Aiken (2003), Harrell (2001), Hosmer and Lemeshow (1989), Kreft and de Leeuw (1998), Myers (1990), Singer and Willett (2003), or other texts on regression methods for more measures to detect and resolve collinearity problems.
Collinearity can change parameter estimates, increase standard errors, and reduce the power to detect reliable effects of correlated variables in a regression model. As demonstrated, collinearity affects only those IVs highly correlated with other IVs, and this may involve a set of IVs. Collinearity does not affect other IVs in the model nor any summary or fit statistics, such as R², AIC, or BIC values.
Critics of an article, such as Pierce et al. (1998), might claim that collinearity should obscure the results and call the conclusions into question. As a retort, however, one would justifiably claim that if collinearity exists, it should apply only to the estimates of the correlated variables, and in any case and by every account (every methodological text cited here) the reported significance values would be an underestimate of the true relationship.