VIF & Remedies in Python

ECON 3209 · Week 11, Lecture 3 · Kerala Agricultural University

Department of Development Economics, KAU

Autumn 2026

Learning Outcomes

By the end of this lecture, you will be able to:

1. compute and interpret variance inflation factors
2. use statsmodels tools to diagnose severe multicollinearity
3. compare VIFs before and after modifying a specification
4. describe practical remedies and their trade-offs
5. preview ridge regression using a matrix formula

Variance Inflation Factor

The VIF for regressor \(X_j\) is

\[\text{VIF}_j = \frac{1}{1 - R_j^2}\]

where \(R_j^2\) comes from regressing \(X_j\) on all the other regressors.

• If \(R_j^2\) is large, \(X_j\) is highly explained by the remaining regressors.
• Higher VIF means more variance inflation in \(\hat\beta_j\).

How Should We Read VIFs?

VIF	Interpretation
1	no inflation
1 to 5	mild to moderate concern
above 5	important warning
above 10	often considered severe

These thresholds are rules of thumb, not universal laws. Economic theory still matters.

Remedy Menu

• Collect more informative data if possible.
• Drop a variable only if it is redundant and not central to the question.
• Combine related measures into an index.
• Center variables when interactions or polynomials create artificial collinearity.
• Use ridge regression when prediction is the main goal.

Ridge Preview

Ridge regression shrinks coefficients by solving

\[\hat\beta^{ridge} = (X'X + \lambda I)^{-1}X'Y\]

• \(\lambda > 0\) stabilizes inversion.
• Bias is introduced intentionally.
• Variance can fall sharply.

Ridge is mainly a prediction tool, not a standard causal estimator.

Python Demo: Compute VIFs

Python Demo: Dropping a Redundant Variable

Centering Can Help with Constructed Collinearity

If we include \(X\) and \(X^2\), or \(D\) and \(D\times X\), collinearity may partly come from construction.

Centering creates \(X_c = X - \bar{X}\) and often improves numerical stability without changing the fitted curve.

Python Demo: Ridge-style Shrinkage with NumPy

Python Demo: OLS vs Ridge Coefficients

Choosing a Remedy

• If the question is causal or explanatory, theory should dominate.
• If two variables measure the same thing, combining them may be sensible.
• If the goal is prediction, shrinkage tools can be helpful.

Never treat VIF as an automatic delete button. It is a diagnostic, not a verdict.

🏋️ Exercise

1. Simulate four regressors with at least one strongly collinear pair.
2. Compute VIFs and identify the biggest problem variable.
3. Drop one regressor and recompute VIFs.
4. Calculate ridge coefficients with a chosen penalty value.

Summary

✅ VIF measures how much multicollinearity inflates the variance of a coefficient.

✅ High VIFs are warnings that regressors overlap strongly in information content.

✅ Possible remedies include redesigning the model, collecting better data, combining variables, or using shrinkage for prediction.

✅ Dropping variables should be guided by economics, not by a mechanical threshold alone.

✅ Ridge regression trades a little bias for greater stability when prediction is the priority.

Next Lecture

Week 12 — Heteroscedasticity

We will cover: - what heteroscedasticity is - why it matters for standard errors - how to detect it - how to use robust SEs and WLS